Метод машинного навчання для створення нових лікарських речовин із заданими властивостями

О. Гурбич

doi:10.24144/2616-7700.2022.40(1).126-145

Автор(и)

О. Гурбич Національний університет "Львівська політехніка", Ukraine https://orcid.org/0000-0002-6821-3390

DOI:

https://doi.org/10.24144/2616-7700.2022.40(1).126-145

Ключові слова:

біологічно активні речовини, нейронна мережа, молекула, машинне навчання, молекулярна структура, молекулярний дескриптор

Анотація

Створення нових біологічно активних речовин є однією із найважливіших проблем фармацевтичної галузі. У цій статті запропоновано метод, у якому поєднуються кілька глибоких нейронних мереж для генерування унікальних молекул із заданими властивостями. Генерування доповнюється виправленням хімічної будови молекул із помилками за допомогою рекурентної нейронної мережі з механізмом уваги. Для створених молекулярних структур проведено аналіз хімічних властивостей та оцінку схожості на лікарські речовини. Запропонований ансамбль дозволяє створювати нові унікальні лікарські речовини, контролюючи ступінь розчинності та інші молекулярні дескриптори.

Посилання

Dickson, M., & Gagnon, J. P. (2004). Key factors in the rising cost of new drug discovery and development. Nat Rev Drug Discov, 3, 417–429. https://doi.org/10.1038/nrd1382 [in English].

Jahan, A., Ismail, M. Y., Sapuan, S. M., & Mustapha, F. (2010). Material Screening and Choosng Methods. Materials and DesignMater, 31, 696–705. https://doi.org/10.1016/j.matdes.2009.08.013 [in English].

Schuhmacher, A., Gassmann, O., & Hinder, M. (2016). Changing R&D models in research-based pharmaceutical companies. Journal of Translational Medicine, 14, 105. https://doi.org/10.1186/s12967-016-0838-4 [in English].

Babiarz, J. C. (2008). In FDA Regulatory affairs. A guide for prescription drugs, medical devices and biologics (2nd ed). Informa Healthcare. New York, 34–45 [in English].

Petrova, E. (2014). Innovation and Marketing in the Pharmaceutical Industry. International Series in Quantitative Marketing 20, Springer-Verlag: New York. https://doi.org/10.1007/978-1-4614-7801-0 [in English].

Kim, S., Thiessen, P. A., Bolton, E. E, Chen, J., Fu, G., Gindulyte, A., Han, L., He, J., He, S., Shoemaker, B. A., Wang, J., Yu, B., Zhang, J., & Bryant, S. H. (2016). PubChem Substance and Compound databases. Nucleic Acids Res, 44(D1), D1202–D1213. https://doi.org/10.1093/nar/gkv951 [in English].

Kirkpatrick, P., Ellis, C. (2004). Chemical space. Nature, 432, 823. https://doi.org/10.1038/432823a [in English].

Bloom, N., Jones, C. I., Van Reenen, J., &Webb, M. (2020). Are Ideas Getting Harder to Find? American Economic Review, 110(4), 1104–1144. https://doi.org/10.3386/w23782 [in English].

LeCunn, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436–444. https://doi.org/10.1038/nature14539 [in English].

Goh, G. B., Hodas, N. O., & Vishnu, A. (2017). Deep learning for computational chemistry. Journal Computational Chemistry, 38, 1291–1307. https://doi.org/10.1002/jcc.24764 [in English].

Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2018). Deep learning for healthcare: review, opportunities and challenges. Briefings in Bioinformatics, 19(6), 1236-1246. https://doi.org/10.1093/bib/bbx044 [in English].

Schneider, G. (2018). Automating drug discovery. Nature Reviews Drug Discovery, 17, 97–113. https://doi.org/10.1038/nrd.2017.232 [in English].

Bostrom, J., Brow, D. G., Young, R. J., & Keseru, G. M. (2018). Expanding the medicinal chemistry synthetic toolboxNat. Nature Reviews Drug Discovery, 17, 709–727. https://doi.org/10.1038/nrd.2018.116 [in English].

Butler, K. T., Davies, D. W., Cartwright, H., Isayev, O., & Walsh, A. (2018). Machine learning for molecular and materials science. Nature, 559, 547–555. https://doi.org/10.1038/s41586-018-0337-2 [in English].

Zhavoronkov, A., Ivanenkov, Y. A., Aliper, A., Veselov, M. S., Aladinskiy, V. A., Aladinskaya, A. V., Terentiev, V. A., Polykovskiy, D. A., Kuznetsov, M. D., Asadulaev, A., Volkov, Y., Zholus, A., Shayakhmetov, R. R., Zhebrak, A., Minaeva, L. I., Zagribelnyy, B. A., Lee, L. H., Soll, R., Madge, D., Xing, L., Guo, T., & Aspuru-Guzik, A. (2019). Deep learning enables rapid identification of potent DDR1 kinase inhibitors. Deep learning enables rapid identification of potent DDR1 kinase inhibitors. Nature biotechnology, 37, 1038–1040. https://doi.org/10.1038/s41587-019-0224-x [in English].

Steinhauser, M. O., & Hiermaier, S. (2009). A Review of Computational Methods in Materials Science: Examples from Shock-Wave and Polymer Physics. International Journal of Molecular Sciences, 10(12), 5135–5216. https://doi.org/10.3390/ijms10125135 [in English].

Behler, J. (2010). Neural network potential-energy surfaces for atomistic simulations. Chemical Modelling: Applications and Theory, 7, 141. https://doi.org/10.1039/9781849730884-00001 [in English].

Ghasemi, S. A., Hofstetter, A., Saha, S., & Goedecker, S. (2015). Interatomic potentials for ionic systems with density functional accuracy based on charge densities obtained by a neural network. Physical review B, 92, 131. https://doi.org/10.1103/PhysRevB.92.045131 [in English].

Schutt, K. T., Arbabzadah, F., Chmiela, S., Muller ,K. R., & Tkatchenko, A. (2017). Quantum-chemical insights from deep tensor neural networks. Nature Communication, 8, 890. https://doi.org/10.1038/ncomms13890 [in English].

Carrasquilla, J., Melko, R. G. (2017). Machine learning phases of matter. Nature Physics, 13, 431–434. https://doi.org/10.1038/nphys4035 [in English].

Xie, T., & Grossman, J. C. (2018). Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties. Physical review letters, 120, 301. https://doi.org/10.1103/Phys-RevLett.120.145301 [in English].

Ryan, K., Lengyel, J., & Shatruk, M. J. (2018). Crystal Structure Prediction via Deep Learning. American Chemical Society Publication, 140(32), 10158–10168. https://doi.org/10.1021/jacs.8b03913 [in English].

Amabilino, S., Bratholm, L. A., Bennie, S. J., Vaucher, A. C., Reiher, M., & Glowacki, D. R. (2019). Training Neural Nets To Learn Reactive Potential Energy Surfaces Using Interactive Quantum Chemistry in Virtual Reality. American Chemical Society Publication, 123(20), 4486-4499. https://doi.org/10.1021/acs.jpca.9b01006 [in English].

Bock, F. E., Aydin, R. C., Cyron, C. J., Huber, N., Kalidindi, S. R., & Klusemann, B. (2019). A Review of the Application of Machine Learning and Data Mining Approaches in Continuum Materials Mechanics. Machine Learning and Data Mining in Materials Science, 6, 110. https://doi.org/10.3389/fmats.2019.00110 [in English].

Haghighatlari, M., & Hachmann, J. (2019). Advances of machine learning in molecular modeling and simulation. Current Opinion in Chemical Engineering, 23, 51–57. https://doi.org/10.1016/j.coche.2019.02.009 [in English].

Chiriki, S., & Bulusu, S. S. (2016). Modeling of DFT quality neural network potential for sodium clusters: Application to melting of sodium clusters (Na20 to Na40). Chemical Physics Letters, 652, 130–135. https://doi.org/10.1016/j.cplett.2016.04.013 [in English].

Shen, L., & Yang, W. J. (2018). Molecular Dynamics Simulations with Quantum Mechanics. Molecular Mechanics and Adaptive Neural Networks. American Chemical Society Publication, 14, 1442–1455. https://doi.org/10.1021/acs.jctc.7b01195 [in English].

Jindal, S., Bulusu, S. S. (2018). A transferable artificial neural network model for atomic forces in nanoparticles. The Journal of Chemical Physics, 149, 101. https://doi.org/10.1063/1.5043247 [in English].

Shweta, J., Satya, S. & Bulusu S. (2018). A transferable artificial neural network model for atomic forces in nanoparticles. Chemical Physics. arXiv:1810.06204 [in English].

Schutt, K. T., Sauceda, H. E., Kindermans, P. J., Tkatchenko, A., & Muller, K. R. (2018). SchNet – A deep learning architecture for molecules and materials. The Journal of Chemical Physics, 148, 722. https://doi.org/10.1063/1.5019779 [in English].

Perez, A., & Martinez-Rosell, G. (2018). Simulations meet machine learning in structural biology. Curr. Opin. Struct. Biol, 49, 139–144. https://doi.org/10.1016/j.sbi.2018.02.004 [in English].

Herr, J., Yao, K., McIntyre, K., Toth, D. W., & Parkhill, J. (2018). Metadynamics for training neural network model chemistries: A competitive assessment. The Journal of Chemical Physics, 148, 241. https://doi.org/10.1063/1.5020067 [in English].

Yao, K., Herr, J. E., Toth, D. W., MckIntyre, R., & Parkhill, J. (2018). The TensorMol-0.1 model chemistry: a neural network augmented with long-range physics. Chemical Science, 9, 2261–2269. https://doi.org/10.1039/C7SC04934J [in English].

Wang, H., Zhang, L., & Han, J. (2018). DeePMD-kit: A deep learning package for many-body potential energy representation and molecular dynamics. Computer Physics Communications, 228, 178–184. https://doi.org/10.1016/j.cpc.2018.03.016 [in English].

Zhang, L., Wang, H., Han, J., & Car, R. (2018). Deep Potential Molecular Dynamics: A Scalable Model with the Accuracy of Quantum Mechanics. Physical review letters, 120, 3001. https://doi.org/10.1103/PhysRevLett.120.143001 [in English].

Zhang, L., & Wang, H. (2018). Adaptive coupling of a deep neural network potential to a classical force field. The Journal of Chemical Physics, 149, 154. https://doi.org/10.1063/1.5042714 [in English].

Zhang, L., Han, J., Wang, H., & Car, R. J. (2018). DeePCG: Constructing coarsegrained models via deep neural networks. The Journal of Chemical Physics, 149, 4101. https://doi.org/10.1063/1.5027645 [in English].

Lusci, A., Pollastri, G., & Baldi, P. J. (2013). Deep Architectures and Deep Learning in Chemoinformatics: The Prediction of Aqueous Solubility for Drug-Like Molecules. American Chemical Society Publications, 53, 1563–1575. https://doi.org/10.1021/ci400187y [in English].

Dahl, G. E. Jaitly, N., & Salakhutdinov, R. (2014). Multi-task Neural Networks for QSAR Predictions. Machine Learning. arXiv:1406.1231 [in English].

Pyzer-Knapp, E. O., Li, K., & Aspuru-Guzik, A. (2015). Learning from the Harvard Clean Energy Project: The Use of Neural Networks to Accelerate Materials Discovery. Advanced Functional Materials, 25, 6495–6502. https://doi.org/10.1002/adfm.201501919 [in English].

Alipanahi, B., Delong, A., Weirauch, M. T., & Frey, B. J. (2015). Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning. Nature Biotechnology, 33, 831-838. https://doi.org/10.1038/nbt.3300 [in English].

Wallach, I., Dzamba, M., & Heifets, A. (2015). AtomNet: A Deep Convolutional Neural Network for Bioactivity Prediction in Structure-based Drug Discovery. Machine Learning. arXiv:1510.02855 [in English].

Mayr, A., Klambauer, G., Unterthiner, T., & Hochreiter, S. (2016). Deep-Tox: Toxicity Prediction using Deep Learning. Frontiers Environmental, 3, 80. https://doi.org/10.3389/fenvs.2015.00080 [in English].

Bjerrum, E. J. (2017). SMILES Enumeration as Data Augmentation for Neural Network Modeling of Molecules. Machine Learning. arXiv:1703.07076 [in English].

Sharma, A. K., Srivastava, G. N., Roy, A., & Sharma, V. K. (2017). ToxiM: A Toxicity Prediction Tool for Small Molecules Developed Using Machine Learning and Chemoinformatics Approaches. Frontiers in Pharmacology, 8, 880. https://doi.org/10.3389/fphar.2017.00880 [in English].

Kearnes, S., Goldman, B., & Pande, V. (2017). Modeling Industrial ADMET Data with Multitask Networks. Machine Learning. arXiv:1606.08793 [in English].

Jimenez, J., Skalic, M., Martinez-Rosell, G., & De Fabritiis, G. J. (2018). KDEEP: Protein–Ligand Absolute Binding Affinity Prediction via 3D-Convolutional Neural Networks. Journal of Chemical Information and Modeling, 58, 287–296. https://doi.org/10.1021/acs.jcim.7b00650 [in English].

Goh, G. B., Hodas, N. O., Siegel, C., & Vishnu, A. (2018). SMILES2Vec: An Interpretable General-Purpose Deep Neural Network for Predicting Chemical Properties. Machine Learning. arXiv:1712.02034 [in English].

Goh, G. B., Siegel, C., Vishnu, A., & Hodas, N. O. (2018). Using Rule-Based Labels for Weak Supervised Learning: A ChemNet for Transferable Chemical Property Prediction. Machine Learning. arXiv:1712.02734 [in English].

Stahl, N., Falkman, G., Karlsson, A., Mathiason, G., & Bostrom, J. J. (2018). Deep Convolutional Neural Networks for the Prediction of Molecular Properties: Challenges and Opportunities Connected to the Data. Journal of Integrative Bioinformatics, 65, 1613–4516. https://doi.org/10.1515/jib-2018-0065 [in English].

Lipinski, C. A., Lombardo, F., Dominy, B. W., & Feeney, P. J. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. Advanced Drug Delivery Reviews, 46, 3–26. https://doi.org/10.1016/S0169-409X(96)00423-1 [in English].

Ghose, A. K., Viswanadhan, V. N., & Wendoloski, J. J. (2013). A Knowledge-Based Approach in Designing Combinatorial or Medicinal Chemistry Libraries for Drug Discovery. A Qualitative and Quantitative Characterization of Known Drug Databases. American Chemical Society Publications, 1, 55–68. https://doi.org/10.1021/cc9800071 [in English].

Egan ,W. J., Merz, K. M., & Baldwin, J. J. (2000). Prediction of Drug Absorption Using Multivariate Statistics. American Chemical Society Publications. American Chemical Society Publications, 43, 3867–3877. https://doi.org/10.1021/jm000292e [in English].

Muegge, I., Heald, S. L., & Brittelli, D. (2001). Simple Selection Criteria for Drug-like Chemical Matter. American Chemical Society Publications, 44, 1841-1846. https://doi.org/10.1021/jm015507e [in English].

Veber, D. F., Johnson, S. R., Cheng, H. Y., Smith, B. R., Ward, K. W., & Kopple, K. D. (2002). Molecular Properties That Influence the Oral Bioavailability of Drug Candidates. American Chemical Society Publications, 45, 2615–2623. https://doi.org/10.1021/jm020017n [in English].

Segler, M. H., Kogej, T., Tyrchan, C., & Waller, M. P. (2018). Synthesis and Cytotoxic Evaluation of Arimetamycin A and Its Daunorubicin and Doxorubicin Hybrids. American Chemical Society Publications, 4(1), 120–131. https://doi.org/10.1021/acscentsci.7b0051z [in English].

Kusner, M. J., Paige, B., & Hernandez-Lobato, J. M. (2017). Grammar Variational Autoencoder. Machine Learning. arXiv:1703.01925v1 [in English].

Goh, G. B., Siegel, C., Vishnu, A., Hodas, N. O., & Baker, N. (2018). How Much Chemistry Does a Deep Neural Network Need to Know to Make Accurate Predictions? Machine Learning. arXiv:1710.02238 [in English].

Goh, G. B., Sakloth, K., Siegel, C., Vishnu, A., & Pfaendtner, J. (2018). Multimodal Deep Neural Networks using Both Engineered and Learned Representations for Biodegradability Prediction. Machine Learning. arXiv:1808.04456 [in English].

Kuzminykh, D., Polykovskiy, D., Kadurin, A., Zhebrak, A., Baskov, I., Nikolenko, S., Shayakhmetov, R., & Zhavoronkov, A. (2018). 3D Molecular Representations Based on the Wave Transform for Convolutional Neural Networks. American Chemical Society Publications, 15, 4378–4385. https://doi.org/10.1021/acs.molpharmaceut.7b01134 [in English].

Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2019). A Comprehensive Survey on Graph Neural Networks. Machine Learning. arXiv:1901.00596v2 [in English].

Zhou, J., Cui, G., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2019). Graph Neural Networks: A Review of Methods and Applications. Machine Learning. arXiv:1812.08434v3 [in English].

Kearnes, S., McCloskey, K., Berndl, M., Pande, V., & Riley, P. (2016). Molecular graph convolutions: moving beyond fingerprints. Journal of Computer-Aided Molecular Design, 30(8), 595–608. https://doi.org/10.1007/s10822-016-9938-8 [in English].

Duvenaud, D., Maclaurin, D., Aguilera-Iparraguirre, J., Gomez-Bombarelli, R., Hirzel, T., Aspuru-Guzik, A. & Adams, R. P. (2015). Automatic chemical design using a data-driven continuous representation of molecules, Advances in Neural Information Processing Systems 28 (NIPS 2015), Montreal, 2016, 2224–2232 [in English].

You, J., Liu, B., Ying, R., Pande, V., & Leskovec, J. (2019). Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation. Machine Learning. arXiv:1806.02473v3 [in English].

Fout, A., Byrd, J., Shariat, B., & Ben-Hur, A. (2017). Composition-Based Multi-Relational Graph Convolutional Networks, Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, 6530–6539 [in English].

Zitnik, M., Agrawal, M., & Leskovec, J. (2018). Modeling polypharmacy side effects with graph convolutional networks. Machine Learning. arXiv:1802.00543v2 [in English].

De Cao, N., Kipf, T. (2018). MolGAN: An implicit generative model for small molecular graphs. Machine Learning. arXiv:1805.11973v1 [in English].

Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B.,Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). A generative adversarial network for artifact removal in photoacoustic computed tomography with a linear-array transducer, Advances in Neural Information Processing Systems 27 (NIPS 2014), Montreal, 2672–2680 [in English].

Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., & Bharath, A. A. (2018). Generative Adversarial Networks: An Overview. IEEE Signal Processing Magazine, 35(1), 53-65. https://doi.org/10.1109/MSP.2017.2765202 [in English].

Jorgensen, P. B., Schmidt, M. N., & Winther, O. (2018). Deep Generative Models for Molecular Science. Molecular Informatic, 37, 133. https://doi.org/10.1002/minf.201700133 [in English].

Gomez-Bombarelli, R., Wei, J. N., Duvenaud, D., Hernandez-Lobato, J. M., Sanchez-Lengeling, B., Sheberla, D., Aguilera-Iparraguirre, J., Hirzel, T. D., Adams, R. P., & Aspuru-Guzik, A. (2018). Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules. American Chemical Society Publications, 4(2), 268-276. https://doi.org/10.1021/acscentsci.7b00572 [in English].

Kadurin, A., Nikolenko, S., Khrabrov, K., Aliper, A., & Zhavoronkov, A. (2017). druGAN: An Advanced Generative Adversarial Autoencoder Model for de Novo Generation of New Molecules with Desired Molecular Properties in Silico. American Chemical Society Publications, 14, 3098–3104. https://doi.org/10.1021/acs.molpharmaceut.7b00346 [in English].

Putin, E., Asadulaev, A., Vanhaelen, Q., Ivanenkov, Y., Aladinskaya, A. V., Aliper, A., & Zhavoronkov, A. (2018). Adversarial Threshold Neural Computer for Molecular de Novo Design. American Chemical Society Publications, 15, 4386–4397. https://doi.org/10.1021/acs.molpharmaceut.7b01137 [in English].

Blaschke, T., Olivecrona, M., Engkvist, O., Bajorath, J., & Chen, H. (2018). Application of Generative Autoencoder in De Novo Molecular Design. Special Issue: Generative Model, 37, 123. https://doi.org/10.1002/minf.201700123 [in English].

Bjerrum, E. J., & Sattarov, B. (2018). Improving Chemical Autoencoder Latent Space and Molecular De Novo Generation Diversity with Heteroencoders. Biomolecules, 8, 131. https://doi.org/10.3390/biom8040131 [in English].

Guimaraes, G., Sanchez-Lengeling, B., Outeiral, C., Farias, P. L. C., & Aspuru-Guzik, A. (2018). Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models. Machine Learning. arXiv:1705.10843v3 [in English].

Dai, H., Tian, Y., Dai, B., Skiena, S., & Song, L. (2018). Syntax-Directed Variational Autoencoder for Structured Data. Machine Learning. arXiv:1802.08786v1 [in English].

Hinton, G. E., & Zemel, R. S. (1993). Autoencoders, Minimum Description Length, and Helmholtz Free Energy, Proceedings of the 6th International Conference on Neural Information Processing Systems (NIPS 1993), Denver, 3–10. [in English].

Kingma, D. P., & Welling, M. (2014). Auto-Encoding Variational Bayes. Machine Learning. arXiv:1312.6114v10 [in English].

eMolecules Announces Version 2.0 of its Chemical Search Engine (2022). Retrieved from https://www.emolecules.com/info/plus/download-database [in English].

Huuskonen, J. J. (2000). Estimation of Aqueous Solubility for a Diverse Set of Organic Compounds Based on Molecular Topology. American Chemical Society Publications, 40, 773–777. https://doi.org/10.1021/ci9901338 [in English].

Hou, T., Xia, K., Zhang, W., & Xu, X. (2004). ADME Evaluation in Drug Discovery, Prediction of Aqueous Solubility Based on Atom Contribution Approach. American Chemical Society Publications, 44, 266-275. https://doi.org/10.1021/ci034184n [in English].

Delaney, J. S. (2004). ESOL: Estimating Aqueous Solubility Directly from Molecular Structure. American Chemical Society Publications, 44, 1000–1005. https://doi.org/10.1021/ci034243x [in English].

DLS-100 Solubility Dataset (2022). Retrieved from https://risweb.st-andrews.ac.uk/. https://doi.org/10.17630/3a3a5abc-8458-4924-8e6c-b804347605e8 [in English].

Llinas, A., Glen, R. C., & Goodman, J. M. (2008). Solubility Challenge: Can You Predict Solubilities of 32 Molecules Using a Database of 100 Reliable Measurements? American Chemical Society Publications, 48, 1289–1303. https://doi.org/10.1021/ci800058v [in English].

Hopfinger, A. J., Esposito, E. X., Llinas, A., Glen, R. C., & Goodman, J. M. (2009). Findings of the Challenge To Predict Aqueous Solubility. American Chemical Society Publications, 49, 1–5. https://doi.org/10.1021/ci800436c [in English].

Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal Artificing Intelligence Recache, 16, 321–357. https://doi.org/10.1613/jair.953 [in English].

Lemaitre, G., Nogueira, F., & Aridas, C. K. (2017). Imbalanced-learn: A python toolbox to tackle the curse of imbalanced datasets in machine learning. The Journal of Machine Learning Research, 18(1), 559–563 [in English].

RDKit: Open-source cheminformatics (2022). Retrieved from http://www.rdkit.org [in English].

Sutskever, I., Vinyals, O., & Le, Q. V. (2015). Sequence to Sequence Learning with Neural Networks, Advances in Neural Information Processing Systems 27 (NIPS 2014), Montreal, 3104–3112 [in English].

Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. Machine Learning. arXiv:1409.0473 [in English].

Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computer, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735 [in English].

Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J. (2013). Distributed Representations of Words and Phrases and their Compositionality, Advances in Neural Information Processing Systems 26 (NIPS 2013), Lake Tahoe, 3111-3119 [in English].

Lamb, A., Goyal, A., Zhang, S., Courville, A. C., & Bengio, Y. (2016). Professor Forcing: A New Algorithm for Training Recurrent Networks, Advances in Neural Information Processing Systems 29 (NIPS 2016), Barcelona, 4601–4609 [in English].

Vincent, P., Larochelle, H. Bengio, Y., & Manzagol, P. (2008). Extracting and composing robust features with denoising autoencoders, Proceedings of the 25th International Conference on Machine Learning, Helsinki, 1096–1103. https://doi.org/10.1145/1390156.1390294 [in English].