Hi @Tiejun_Cheng and welcome to our community!
Thank you for the detailed bug report. After your message, we’ve identified that the root cause is in the LLM extraction input data. Some responses from the OpenAI Batch API returns corrupted JSON escape sequences (\u0000 instead of \u00 for accented characters like é, ö, ® as you have noted), which json.loads() then interprets as embedded null bytes.
This affects:
- 171 rows in investigated_drugs.synonyms that we use for
drug_molecule - 6 rows in diseases.diseaseFromSource in
clinical_report - … and **
evidence_clinical_precedence**andclinical_targetbecause they propagate the malformed disease names we see in the above datasets
We will prepare a fix for our next release that patches the intended characters from the corrupted escape sequences (e.g., \x00e9 → é). In the meantime, we hope you can omit or work around these affected rows on your end. We apologise for the inconvenience. Please let us know if you find any other issues.
Best,
Irene