OSB looks at the response-code in the Response Metadata to flag an application error. If this value is a non-zero, then it is treated as an application error.
The easiest way to manipulate this value is to add a Reply with Failure in the OSB service you are calling.
You can also manipulate the response code by inserting a response-code element in $inbound as shown below. Note: This must be done in the response pipeline
Note: I only researched this for Business Services that were calling OSB Proxy Services as that was the architecture for the context of the question asked. More research would need to be done to determine this behavior when call services outside the OSB.