Weird characters in compound synonym

Community Feedback
1

To whom it may concern,

I found some weird characters in the compound synonyms, e.g., verorab�0000ae on the following page. Note that you have to click “show more” to see this name. I have seen more cases like this in the download file: Index of /pub/databases/opentargets/platform/26.06/output/drug_molecule

Sincerely
Tiejun Cheng

2

Hi @Tiejun_Cheng and welcome to our community!

Thank you for the detailed bug report. After your message, we’ve identified that the root cause is in the LLM extraction input data. Some responses from the OpenAI Batch API returns corrupted JSON escape sequences (\u0000 instead of \u00 for accented characters like é, ö, ® as you have noted), which json.loads() then interprets as embedded null bytes.

This affects:

  • 171 rows in investigated_drugs.synonyms that we use for drug_molecule
  • 6 rows in diseases.diseaseFromSource in clinical_report
  • … and **evidence_clinical_precedence**and clinical_target because they propagate the malformed disease names we see in the above datasets

We will prepare a fix for our next release that patches the intended characters from the corrupted escape sequences (e.g., \x00e9 → é). In the meantime, we hope you can omit or work around these affected rows on your end. We apologise for the inconvenience. Please let us know if you find any other issues.

Best,
Irene

3

Thanks for the prompt response. No rush. I just ignore such characters for now.

Best
Tiejun