Hi @qwang and welcome to our Community!
The Clinical Precedence evidence in the 26.06 release introduced a new method to extract drug indications from clinical trials, which explains the differences you observe. The new method consists of replacing the disease and drug annotations provided in the clinical trial with entities curated by an LLM. The motivation for this is that the raw annotation was often context-insensitive, leading to false associations. You can learn more about it in our documentation or our working ticket describing the effort.
This means that the entity previously linked to the trial studying Gabapentin, for example, will vary, however the signal is still there. Looking at your example, CACNA1H is still linked to alcohol drinking only that the disease ID has changed as a result of the LLM extraction.
We understand that keeping track of all ontology/mapping changes is challenging. If you’re interested in targets related with alcohol consumption, I’d suggest you take into account an array of disease IDs. I suggest picking these 7 IDs instead of relying on EFO_0004329 only.
┌───────────────┬───────────┬──────────────┬───────────────────────────────────────────────────────────────────────────────┐
│ diseaseId ┆ evd_count ┆ target_count ┆ diseaseFromSource │
│ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ u32 ┆ u32 ┆ list[str] │
╞═══════════════╪═══════════╪══════════════╪═══════════════════════════════════════════════════════════════════════════════╡
│ MONDO_0007079 ┆ 2557 ┆ 140 ┆ ["alcoholism", "alcohol dependence", … "alcohol dependency"] │
│ MONDO_0005433 ┆ 324 ┆ 79 ┆ ["alcohol withdrawal", "alcohol withdrawal syndrome"] │
│ MONDO_0002046 ┆ 758 ┆ 128 ┆ ["alcohol use disorder", "alcohol use disorders", … "alcoholic use disorder"] │
│ EFO_0004329 ┆ 18 ┆ 15 ┆ ["alcohol consumption"] │
│ MONDO_0012454 ┆ 3 ┆ 3 ┆ ["alcohol intolerance"] │
│ HP_0030955 ┆ 102 ┆ 79 ┆ ["alcoholism", "alcohol addiction"] │
│ MONDO_0021698 ┆ 3 ┆ 3 ┆ ["alcohol-related disorders"] │
└───────────────┴───────────┴──────────────┴───────────────────────────────────────────────────────────────────────────────┘
Source code
(evd.filter(
# Get evidence where disease label mentions alcohol
pl.col("diseaseFromSource").str.to_lowercase().str.contains("alcohol")
).filter(
# And filter out unrelated diseases
~pl.col("diseaseFromSource").str.to_lowercase().str.contains("hepatitis")
).filter(
~pl.col("diseaseFromSource").str.to_lowercase().str.contains("liver")
).filter(
~pl.col("diseaseFromSource").str.to_lowercase().str.contains("cirrhosis")
).group_by(
"diseaseId"
).agg(
pl.len().alias("evd_count"),
pl.col("targetId").unique().len().alias("target_count"),
pl.col("diseaseFromSource").unique(),
))
If you follow this approach, this yields 152 associated targets. This number was 156 for the previous release, so I looked at the root cause for the difference:
- 13 targets are no longer associated with alcohol consumption.
┌─────────────────┬─────────────┬────────────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────┐
│ targetId ┆ old_reports ┆ new_conditions ┆ qualityControls │
│ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ str ┆ list[struct[2]] ┆ list[str] │
╞═════════════════╪═════════════╪════════════════════════════════════════════════════════════════════╪════════════════════════════════════════════════════════╡
│ ENSG00000109738 ┆ nct00000261 ┆ null ┆ ["INDIRECT_PRIMARY_PURPOSE", "NO_DISEASE"] │
│ ENSG00000196517 ┆ nct00764660 ┆ [{"heavy drinking",null}, {"relapse","EFO_0004952"}] ┆ ["INDIRECT_PRIMARY_PURPOSE", "UNVALIDATED_INDICATION"] │
│ ENSG00000145888 ┆ nct00000261 ┆ null ┆ ["INDIRECT_PRIMARY_PURPOSE", "NO_DISEASE"] │
│ ENSG00000168412 ┆ nct03043443 ┆ [{"sleeping problems",null}] ┆ null │
│ ENSG00000082482 ┆ nct00000261 ┆ null ┆ ["INDIRECT_PRIMARY_PURPOSE", "NO_DISEASE"] │
│ ENSG00000169836 ┆ nct01181908 ┆ null ┆ ["INDIRECT_PRIMARY_PURPOSE", "NO_DISEASE"] │
│ ENSG00000169427 ┆ nct00000261 ┆ null ┆ ["INDIRECT_PRIMARY_PURPOSE", "NO_DISEASE"] │
│ ENSG00000186795 ┆ nct00000261 ┆ null ┆ ["INDIRECT_PRIMARY_PURPOSE", "NO_DISEASE"] │
│ ENSG00000012504 ┆ nct02654236 ┆ [{"liver injury",null}] ┆ ["INDIRECT_PRIMARY_PURPOSE"] │
│ ENSG00000157103 ┆ nct01904487 ┆ [{"healthy control",null}, {"alcohol dependence","MONDO_0007079"}] ┆ ["INDIRECT_PRIMARY_PURPOSE"] │
│ ENSG00000171303 ┆ nct00000261 ┆ null ┆ ["INDIRECT_PRIMARY_PURPOSE", "NO_DISEASE"] │
│ ENSG00000134640 ┆ nct03043443 ┆ [{"sleeping problems",null}] ┆ null │
│ ENSG00000100433 ┆ nct00000261 ┆ null ┆ ["INDIRECT_PRIMARY_PURPOSE", "NO_DISEASE"] │
└─────────────────┴─────────────┴────────────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────┘
old_reports has the NCT ID for the trial supporting the association. new_conditions is the extracted diseases with the new pipeline and qualityControls is our QC flags for this trial with the new pipeline. The source of difference is also due to the LLM extraction work, where on top of extracting entities, we extract the purpose of the intervention not to derive associations from trials where the intervention is not directly treating the disease.
Source code
old_targets = (
old_evd.filter(
pl.col("diseaseFromSource").str.to_lowercase().str.contains("alcohol")
)
.filter(~pl.col("diseaseFromSource").str.to_lowercase().str.contains("hepatitis"))
.filter(~pl.col("diseaseFromSource").str.to_lowercase().str.contains("liver"))
.filter(~pl.col("diseaseFromSource").str.to_lowercase().str.contains("cirrhosis"))
.group_by("targetId")
.agg(pl.col("clinicalReportId").unique().alias("old_reports"))
)
targets = (
evd.filter(pl.col("diseaseFromSource").str.to_lowercase().str.contains("alcohol"))
.filter(~pl.col("diseaseFromSource").str.to_lowercase().str.contains("hepatitis"))
.filter(~pl.col("diseaseFromSource").str.to_lowercase().str.contains("liver"))
.filter(~pl.col("diseaseFromSource").str.to_lowercase().str.contains("cirrhosis"))
.group_by("targetId")
.agg(pl.col("clinicalReportId").first().alias("reports"))
)
(
old_targets.join(targets, "targetId", "anti")
.explode("old_reports")
.join(
cr.select(
pl.col("id").alias("old_reports"),
pl.col("diseases").alias("new_conditions"),
"qualityControls",
),
"old_reports",
"left",
)
)
- 9 new targets are now associated with alcohol consumption.
┌─────────────────┬─────────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬────────────────────────────┐
│ targetId ┆ reports ┆ old_conditions ┆ old_qualityControls │
│ --- ┆ --- ┆ --- ┆ --- │
│ str ┆ str ┆ list[struct[2]] ┆ list[str] │
╞═════════════════╪═════════════╪══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╪════════════════════════════╡
│ ENSG00000187758 ┆ nct00661141 ┆ [{"aldehyde dehydrogenase-2 (aldh2) deficiency",null}] ┆ null │
│ ENSG00000196616 ┆ nct00661141 ┆ [{"aldehyde dehydrogenase-2 (aldh2) deficiency",null}] ┆ null │
│ ENSG00000087085 ┆ nct05042102 ┆ [{"alcohol use disorder","MONDO_0002046"}] ┆ ["UNVALIDATED_INDICATION"] │
│ ENSG00000010310 ┆ nct06939088 ┆ [{"schizophrenia and schizophrenia spectrum psychosis",null}, {"schizophrenia disorders",null}, … {"schizophrenia","MONDO_0005090"}] ┆ ["UNVALIDATED_INDICATION"] │
│ ENSG00000160870 ┆ nct00878306 ┆ [{"drug abuse","EFO_0003890"}, {"hiv infections","EFO_0000764"}] ┆ ["UNVALIDATED_INDICATION"] │
│ ENSG00000021461 ┆ nct00878306 ┆ [{"drug abuse","EFO_0003890"}, {"hiv infections","EFO_0000764"}] ┆ ["UNVALIDATED_INDICATION"] │
│ ENSG00000248144 ┆ nct00661141 ┆ [{"aldehyde dehydrogenase-2 (aldh2) deficiency",null}] ┆ null │
│ ENSG00000106258 ┆ nct00878306 ┆ [{"drug abuse","EFO_0003890"}, {"hiv infections","EFO_0000764"}] ┆ ["UNVALIDATED_INDICATION"] │
│ ENSG00000160868 ┆ nct00878306 ┆ [{"drug abuse","EFO_0003890"}, {"hiv infections","EFO_0000764"}] ┆ ["UNVALIDATED_INDICATION"] │
└─────────────────┴─────────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴────────────────────────────┘
The new targets mostly come from trials that we were previously leaving out because we couldn’t validate the indication. We don’t apply this filter anymore because we trust the newly extracted associations. And a few other targets like ENSG00000187758 come from trials from which we couldn’t assign a disease ID to the condition.
Source code
(
targets.join(old_targets, "targetId", "anti").join(
old_cr.select(
pl.col("id").alias("reports"),
pl.col("diseases").alias("old_conditions"),
pl.col("qualityControls").alias("old_qualityControls"),
),
"reports",
"left",
)
)
I hope this helped you understand the implications of these recent changes. The data is not perfect now, and you might find examples where we have actually lost a valid association. The important bit here is that we have reduced the noise in our data compared to our previous release, and improved the confidence in the associations we do have. Please let us know if you find anything else! We plan to continue refining the pipeline for better value and hope to bring minimal disruption.
Best,
Irene