Hello howertoneng,
Thank you for your letter.
Unfortunately it is not possible to record directly to e.g. VOB format.
But you can do it without the reencoding by the following way:
1) Install DirectShow MPEG-2 compressor and set it in HyperCam options.
2) Then change the format of file using your dvd authoring software (e.g. Nero), to change it e.g. from ASF to VOB. It will change the file container, but not the data of the file (video) itself.