Return values from hooking functions that are meaningful

[ptrstr]

Currently the hooking functions return 0 on success and 1 on failure. It would be pretty useful to have different values for 1 to indicate the type of error.