
==== Front
bioRxiv
BIORXIV
bioRxiv
2692-8205
Cold Spring Harbor Laboratory

10.1101/2024.08.16.608180
preprint
1
Article
Quantized multi-task learning for context-specific representations of gene network dynamics
Chen Han http://orcid.org/0000-0002-1722-8736

Venkatesh Madhavan S http://orcid.org/0000-0002-2698-367X

Gomez Ortega Javier http://orcid.org/0000-0001-7821-4760

Mahesh Siddharth V
Nandi Tarak N
Madduri Ravi K http://orcid.org/0000-0003-2130-2887

Pelka Karin http://orcid.org/0000-0003-2856-8602

Theodoris Christina V http://orcid.org/0000-0003-1658-1447

19 8 2024
2024.08.16.608180https://creativecommons.org/licenses/by-nc-nd/4.0/ This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which allows reusers to copy and distribute the material in any medium or format in unadapted form only, for noncommercial purposes only, and only so long as attribution is given to the creator.
http://biorxiv.org/lookup/doi/10.1101/2024.08.16.608180
nihpp-2024.08.16.608180.pdf
While often represented as static entities, gene networks are highly context-dependent. Here, we developed a multi-task learning strategy to yield context-specific representations of gene network dynamics. We assembled a corpus comprising ~103 million human single-cell transcriptomes from a broad range of tissues and diseases and performed a two stage pretraining, first with non-malignant cells to generate a foundational model and then with continual learning on cancer cells to tune the model to the cancer domain. We performed multi-task learning with the foundational model to learn context-specific representations of a broad range of cell types, tissues, developmental stages, and diseases. We then leveraged the cancer-tuned model to jointly learn cell states and predict tumor-restricting factors within the colorectal tumor microenvironment. Model quantization allowed resource-efficient fine-tuning and inference while preserving biological knowledge. Overall, multi-task learning enables context-specific disease modeling that can yield contextual predictions of candidate therapeutic targets for human disease.
==== Body
pmc
