Full Report
OpenText Directory Services (OTDS) is a Java web application that provides authentication and user management for OpenText applications. They had seen this in various client projects and decided to take a closer look. Upon reviewing the code, nothing obvious came up besides a potential deserialization sink. The function cookieToMap had the exact traits of a deserialization-to-RCE bug: using readObject() on user-controlled data. Except, there was a catch: the cookie must be signed. So, this seems like a dead end... the entire post is about their method for exploiting this. The signed data is generated using an HMAC-SHA1 hash with a controlled message and IV. The message and IV were user-controlled and concatenated together to create the HMAC. The cookie had the following format: 2 bytes for the length of the signature Signature data Length of IV (2 bytes) IV. Length of message (2 bytes) Message. A serialized Java object. The concatenation had no separation between the IV and the message. So, AA,BB is the same as A,ABB. If a part of the message could contain a deserialization payload, then this could be turned into RCE! So, what's the catch? There's a lot of controllable data in the cookie, but it must be valid UTF-8 strings. Additionally, the HMAC was using the compressed message rather than the actual message. This sounds like a CTF! The compression library was zlib. For each implementation of the deflate algorithm, there's a table that must be provided. There is nothing that mandates the exact bytes and table to be used. So, you can just use a table from only 0x01 to 0x7F; there's even a tool called ascii-zip that already does this! So, they generated a payload with ysoserial and compressed it with ascii-zip. Now, this just had to land in the middle of the message. They ended up needing to create their own encoder due to constraints with zlib, thinking the data was uncompressible. They ended up encoding everything by hand due to the complexity of the requirements. After encoding everything by hand, they got a DNS request to work! They were able to use some of the data as a signing oracle, which was super interesting to me. Overall, a great, technical post on hacking software under real constraints.
Analysis Summary
Based on the technical research provided, here is the summary of the vulnerability found in OpenText Directory Services (OTDS).
# Vulnerability: Unsafe Java Deserialization via Cryptographic Signature Bypass
## CVE Details
- **CVE ID:** Not explicitly listed in the text (The article describes a novel discovery by Searchlight Cyber researchers).
- **CVSS Score:** Estimated 9.8 - 10.0 (**Critical**)
- **CWE:** CWE-502 (Deserialization of Untrusted Data), CWE-347 (Improper Verification of Cryptographic Signature)
## Affected Systems
- **Products:** OpenText Directory Services (OTDS)
- **Versions:** Affected versions up to the date of the research (February 2026).
- **Configurations:** Default configuration; the vulnerability is accessible without authentication.
## Vulnerability Description
The vulnerability exists in the `cookieToMap` method within the OTDS Java application. The application attempts to protect a serialized Java object stored in a cookie by using an HMAC-SHA1 signature. However, the implementation contains a critical "length extension-like" flaw in how it handles the Initialization Vector (IV) and the message.
The application concatenates the IV and the message before verifying the HMAC, but it does not use a separator. Because the lengths of the IV and the message are provided by the user in the cookie header, an attacker can shift bytes between the IV and the message. Since the HMAC is calculated on the combined data, the signature remains valid even if bytes are moved.
Furthermore, the message is zlib-compressed. By using specialized encoding techniques (such as "ascii-zip"), researchers bypassed UTF-8 string constraints and used the application as a signing oracle to craft a validly signed, malicious serialized object.
## Exploitation
- **Status:** PoC developed by researchers (DNS request confirmed); no reported exploitation in the wild at the time of the post.
- **Complexity:** High (Requires manual zlib Huffman table manipulation and bypass of UTF-8 constraints).
- **Attack Vector:** Network (Unauthenticated via HTTP Cookies).
## Impact
- **Confidentiality:** High (Full system access/data theft).
- **Integrity:** High (Full system compromise).
- **Availability:** High (Potential for total service disruption).
## Remediation
### Patches
- Users should update to the latest version of OpenText Directory Services provided by the vendor. (Specific version numbers should be verified via the OpenText MySupport portal).
### Workarounds
- Ensure OTDS is not exposed to the public internet unless necessary.
- Implement Web Application Firewall (WAF) rules to inspect and filter unusually large or malformed authentication cookies.
## Detection
- **Indicators of Compromise:** Unusual DNS queries originating from the OTDS server (specifically to OOB interaction platforms like `oastify[.]com`).
- **Detection Methods:** Monitor logs for `OtdsException: invalid content` which may indicate failed exploitation attempts. Review network traffic for large, high-entropy cookies being sent to the `cookieToMap` related endpoints.
## References
- [https://www[.]slcyber[.]io/research/almost-impossible-java-deserialization-through-broken-crypto-in-opentext-directory-services]
- [https://www[.]opentext[.]com/products/directory-services]