🚀 The feature, motivation and pitch

When using the C shim functions provided by the Stable API it is currently impossible to get information about why a particular function returned AOTI_TORCH_FAILURE. A backtrace is printed to stderr, but there's no way to retrieve this programatically. This prevents consumers from providing detailed error messages.

See also https://github.com/pytorch/pytorch/issues/174507#issuecomment-4150977835 where I originally raised this, and @janeyx99 's comment below it that it could be improved.

Alternatives

In my original comment, I stated I expected something like aoti_torch_recent_error_in_thread(char*, size_t len); write the most recent error message to a (provided) buffer. I've since done some exploration & prototyping:

Currently the exception is caught here and logged to stderr. That appears to be an internal header and modifying it to also contain an catch (const c10::Error& e) clause provides us with easy access to the useful information. I think the methods what and what_without_backtrace are the most relevant ones. The what_without_backtrace() value would be what I'd consider the useful error message, while what() with the full backtrace may be good to keep as the in-depth information that helps developers figure out what actually happened on the other side of the C shims.

I originally explored something akin to how XSetErrorHandler works in X11, so something where you can register an error handler and that error handler is called with an handle from which information can be retrieved. Something like this set of functions.

The default callback can perform the current behaviour of logging directly to stderr. But because the callback would be called from a catch block, we don't actually gain much from having the callback (the stack doesn't provide information all the way to where the error occurred), which is what I was hoping to get from this approach. It's very helpful for debugging to get the correct stack to the source of the error, but that only works if the callback would be triggered from the source of the error, not from the catch as we would have in our situation.

This is also a lot of extra additions & complexity, for arguably minimal gains for the stable API interface. So while a good first exploration, I'm not sold on it.

As an alternative, I explored (in 6206ee142bc4ccb4749791646ce05d2bc368a8b5), we could simplify this all to just two functions:

/// Retrieve a pointer to the string that holds the most recent exception's message.
/// This pointer is a borrowed pointer and is invalidated when the next exception occurs.
AOTI_TORCH_EXPORT const char* aoti_torch_exception_get_what();

/// Retrieve a pointer to the string that holds the most recent exception's message and backtrace.
/// This pointer is a borrowed pointer and is invalidated when the next exception occurs.
/// This may be the same as the less detailed aoti_torch_exception_get_what() in case more information
/// is not available.
AOTI_TORCH_EXPORT const char* aoti_torch_exception_get_what_with_backtrace();

To make this work, we create two new thread_local std::string in csrs/shim_common.cpp like this here and modify the try-catch block to populate those. It's simple, avoids overcomplicating and could be extended with additional methods that allow retrieving other aspects of the exception in the future.

I'm not sure yet how we'd use it on the c++ side though; the TORCH_ERROR_CODE_CHECK macro is defined in the 'headeronly' headers, and we'd have to do a forward declaration on aoti_torch_exception_get_what() if we wanted to use that, but then we can hardly call the header headeronly. So in this branch I added TORCH_ERROR_CODE_CHECK_DETAILED to torch/csrc/stable/macros.h instead...

The TORCH_ERROR_CODE macro is used in many places, avoiding changing its behaviour is probably desired, so perhaps we need to do something with a global / thread local / environment flag to toggle the printing behaviour of the backtrace. Not sure what's best here.

There may also be ways to convey the information through the C shims I haven't thought of, obtaining the information seems pretty straightforward because of c10::Error, main choices are how it is exposed through the stable API and how consume it.

Additional context

No response

cc @chauhang @penguinwu @avikchaudhuri @zhxchen17 @tugsbayasgalan @angelayi @suo @ydwu4 @desertfire @yushangdi @benjaminglass1 @jataylo @iupaikov-amd