Daniel Lemire

Daniel Lemire is a full professor of computer science. He is particularly interested in software performance and indexing techniques in data science. He likes to take a critical look at the use of technology.

Daniel Lemire is among the 2% most cited scientists in the world (Stanford University rankings, 2023). He is among the 0.0006% most followed programmers in the world on GitHub; GitHub has over 100 million developers.

Much of our recent technological progress, in all fields, is based on software. Adopted by computer giants, professor Lemire's work has led to remarkable tangible gains:

• Today's disks and networks are capable of transmitting data at gigabytes per second. Unfortunately, our software often artificially limits performance to megabytes per second. This is particularly the case when web services exchange billions of data every day. Professor Lemire developed the first software library called the simdjson parser, capable of processing data from web services (JSON) at gigabytes per second. Today, this parser is used in major systems such as Facebook Velox, and by major companies including Google, Shopify and Intel. It is also part of a fundamental tool in computer science (Node.js), where it helps to load configuration files faster. To date, the simdjson parser remains the world's fastest for processing JSON documents. This high-impact discovery earned him the Université du Québec's Prix d'excellence 2020 for research success in all fields, in a population of over 3,500 researchers. The results of his work are used to speed up parsing in the Google Chrome and Safari browsers. The article On?demand JSON: A better way to parse documents? was the most read article of the last 5 years at Software: Practice and Experience (2024).
• Professor Lemire designed Roaring Bitmaps, which have become a standard and part of countless major systems, including Google, for data analysis within YouTube, as well as for companies such as Uber, Microsoft and Wikipedia. Professor Lemire's work on Roaring bitmaps was conceived as an effective alternative to their former EWAH format, which remains widely used by millions of programmers every day, opening up the scope of this discovery all the more.
• Numbers are usually stored on disks or exchanged over a network as strings, converted into standard binary form by software during operations. This conversion represents a problem that has remained virtually untouched in the scientific literature for almost 30 years. Thanks to a new algorithm, Professor Lemire succeeded in multiplying the speed of number reading within software systems by a factor of four. The algorithm has been adopted by several programming languages (C#, Go and Rust), the standard C++ library under Linux, and several major systems including the Safari, Chrome and Microsoft Edge web browsers. By speeding up a fundamental operation for software systems, this algorithm has become ubiquitous in our everyday computing tools.
• Still on the subject of string operations, Professor Lemire's team has also produced the simdutf software library, which transforms and validates strings six to ten times faster than conventional methods. This library includes several new algorithms, and is part of the popular Node.js JavaScript runtime. In turn, Node.js is used by a wide range of systems, including to create web applications at Netflix, Uber, LinkedIn, Walmart, and is the core engine of major office systems, such as Slack, Discord and Microsoft Teams. His Unicode string validation algorithm is used by several major systems, including the PHP interpreter.

Professor Lemire has also made a wide range of contributions to fundamental computer systems. For example, he produced a new algorithm used by the Linux kernel and by the standard libraries of several programming languages (GNU libstdc++, Microsoft C++ standard library, etc.), in turn used for software operations in a very wide variety of systems. It has broken records for speed and efficiency when decoding and encoding binary data into text (base64), a process involved in the e-mail standard in particular, in some cases reaching speeds 20 times faster than a conventional approach. These algorithms have become popular technologies for web development.

The international impact of his work is also demonstrated by the fact that his algorithms and tools are cited in 37 patents held by companies including Microsoft, LinkedIn, Oracle and Fujitsu Limited.

Professor Lemire also stands out for his work in knowledge mobilization and his communication skills. Since 2004, he has maintained a computer blog, publishing several posts a month. The blog is followed by over 12,500 subscribers from all over the world. His posts are often commented on, enabling him to engage in conversation with the international community. They are cited by other reference blogs and recognized by Silicon Valley, increasing his influence.

Professor Lemire regularly participates, as a member of program committees, in the organization of leading international computer science conferences (e.g. ACM CIKM, WWW, ACM RecSys, etc.). As a speaker, he was named the most popular speaker at QCon2019, which has been viewed over 69,000 times on YouTube to date.

He is also editor of Software: Practice and Experience magazine since 2020. Founded in 1971, this journal is one of the most prestigious in computer science, ranking in the top 20% of most-read and cited journals in the field, with a citation rate that has risen sharply over the past five years. In addition to serving on numerous selection committees for major research funding agencies in Quebec and Canada, he was co-chair of NSERC's Computer Science Discovery Grants Committee in 2020-2021.

Professor Lemire has maintained an average of more than four publications per year over the past ten years, nearly half of which are the result of international collaboration. His publications, cited more than 5,000 times by peers, extend beyond the field of computer science, with applications in the social sciences, psychology and earth sciences. His research work is also cited in several social media. A fervent advocate of free software and the democratization of knowledge, Professor Lemire has widely published the results of his work in open access (open source), including his computing tools, making them freely available to the international computing community.

In short, Professor Lemire has contributed to the training of new researchers with diverse profiles who are still working in research. He also has a remarkable influence as a science popularizer, through his open science approach and his many knowledge transfer initiatives.

Talks (YouTube)

NodeConf EU 2023

BID 2023

SPIRE 2021

Go Systems (San Francisco, 2020)

Performance Summmit III (Seattle, 2020)

QCon San Francisco 2019 (best voted talk!)

Spark Summit East 2017

Laboratory

We are lucky to have a fully equipped laboratory with a dedicated technician. We have a server farm that has been used worldwide for experiments in software performance (e.g., by researchers such as Agner Fog). Some of our machines have the following specifications:Microarchitecture Icelake : Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz

• Icelake microarchitecture: Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz
• Haswell microarchitecture: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
• Knights Landing microarchitecture: Intel(R) Xeon Phi(TM) CPU 7210 @ 1.30GHz (64 cores)
• Skylake microarchitecture: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
• Skylake-X microarchitecture: Intel(R) Xeon(R) W-2104 CPU @ 3.20GHz
• IBM POWER9 2.2 GHz, 4-core
• Cannonlake microarchitecture: Intel Core i3-8121U CPU @2.20GHz
• Skylark microarchitecture (ARMv8): Ampere eMAG CPU 32 cores @ 3.3 GHz

We also have several workstations and space in our laboratory to explore virtual reality as a tool in data science.

Students and post-doctoral fellows

We are recruiting students and postdoctoral fellows. If you love writing crazily fast software and want to come to Montreal, drop us a line. Link to an impressive GitHub profile is an asset. Speaking French is necessary if you want to pursue an academic program with me at the Université of Québec, except maybe at the Ph.D. level where allowances can be made for strong students.  Some of our best students are women. We offer scholarships for graduate studies in software performance for data engineering (in French).

If you are a Canadian undergraduate student with at least a B average, you might be interested in coming to work with Daniel Lemire under an NSERC Undergraduate Student Research Awards. The awards help pay for a full time research project in our Montreal labs. The application deadlines are:

• March 1st for the Summer term;
• July 1st for the Fall term;
• November 1st for the Winter term.

It is an ongoing competition: we receive applications for every term. It is ok if you do not speak French. Please allow at least a week to put together an application with my help.

If you are interested in pursuing a master in information technology full-time under the supervision of Daniel Lemire in Montreal and you know some French, I take applications for NSERC Graduate Scholarships. You need to have a strong academic profile to apply. You should be a Canadian citizen or permanent resident of Canada. The deadline is December first of each year. You must plan ahead. We take applications every year.

If you are interested in pursuing a Ph.D. in cognitive computing full-time under the supervision of Daniel Lemire in Montreal and you know some French, we take applications for NSERC graduate scholarships. You need to have a strong academic profile to apply. You should be a Canadian citizen or permanent resident of Canada. The deadline is November 1st of each year. You must plan ahead. We take applications every year.

Moreover, all students finishing an M.Sc. thesis in information technology with us receive a scholarship, automatically. All students making progress on a doctorate in cognitive computer science receive automatic scholarships. Enrolment in a PhD program implies a waiver of tuition fees for foreign students. 

Daniel Lemire regularly supervises students, from the undergraduate to the Ph.D. level. He works primarily with students who love to program and who prefer an open source setting (e.g., Linux). Many of his students contribute to open-source projects on sites such as GitHub.

He recently supervised the following Ph.D. students: 

• Pierre Marie Ntang, cognitive computer science (graduated in 2023);
• Gary Germeil, cognitive computer science (graduated in 2022);
• Tarek Khei, cognitive computer science (graduated in 2020);
• Xueping Dai, environmental science (gradauted in 2019);
• Erick Aokou Koffi, cognitive computer science (graduated in 2018);
• Badis Merdaoui, cognitive computer science (graduated in 2017);
• Jing Li, computer science (graduated in 2016);
• Samy Chambi, computer science (graduated in 2016);
• Hazel Webb, computer science (graduated in 2010).

Two of his Master's students have been awarded the Governor General's Gold Medal (Verret in 2022, Courcot in 2023). Several of his students occupy key positions: e.g., Maxime Boisvert (M.Sc., 2017) is Production Engineering Manager at Shopify, Shany Carle (M.Sc., 2017) and Carine Croteau (M.Sc., 2020) are computer science professors at cégep de Victoriaville, Shira Smith is engineer at Discord in California.

Books

• Java pas à pas

  

• Programmation avec Python: des jeux au Web

  

• La science des données: Théorie et applications avec R et Python

  

Education

• Postdoctorate (Institute of Biomedical Engineering)
• Engineering Mathematics Ph.D. (University of Montreal and Polytechnique Montréal)
• Master in Mathematics (University of Toronto)
• Bachelor degree in Mathematics (University of Toronto), with High Distinction

Research Interests

• Data Science
• Data Indexing
• Data Engineering
• Software Performance
• Vectorization (SIMD)