Call for papers
We invite submissions of extended abstracts with a maximum length of two pages ACM style.
List of topics for submissions:
- Data collection and benchmark development for legal IR
- Evaluation in the legal domain
- Text processing for legal IR
- Ranking models tailored to the legal domain
- Search UI/UX for the legal domain
- eDiscovery
- Case law retrieval
- Expert finding in legal (e.g., finding the right lawyer for a case)
- Legal Question Answering
- Legal Knowledge Graphs
- Bibliometric-enhanced legal information retrieval
- Other IR tasks in the legal domain
Abstracts may describe relevant contributions (scientific or practical), work in progress, or summarize previously published work in high-quality venues that has a high relevance to the workshop. We also warmly welcome submissions from industry. In case of submitting an abstract about previously published research, please refer to the original publication in your abstract.
You don't need to anonymize your abstract; the reviewing process will be single blind. The selection of papers is based on relevance to the workshop and quality of the research. Since extended abstracts might summarize previously published work, novelty is not the main criterion. We encourage participants to join us on-site.
Please format your abstract using LaTeX. You can use this Overleaf template.
Important dates
- Submission deadline: January 20, 2023
- Notification:
February 17March 3rd, 2023 - Workshop date: April 2nd, 2023
Submission URL (EasyChair): https://easychair.org/my/conference?conf=legalir2023
Organization
- Suzan Verberne, Leiden Institute of Advanced Computer Science, Leiden University, The Netherlands
- Evangelos Kanoulas, Informatics Institute, University of Amsterdam, The Netherlands
- Gineke Wiggers, eLaw Center for Law and Digital Technologies, Leiden University, The Netherlands
- Florina Piroi, Institute of Information Systems Engineering, TU Wien, Austria
- Arjen P. de Vries, Institute for Computing and Information Sciences, Radboud University, The Netherlands
Programme
| Time | Session | Speaker(s) | Title | ||
| 9:00 | Opening | ||||
| 9:10 | Keynote | Milda Norkute | Evaluation of legal search from the end users’ perspective - current challenges and opportunities in industry | ||
| 10:00 | Oral session | David Lewis | Implicit Assumptions in the Evaluation of One-Phase Technology-Assisted Review | ||
| Behrooz Mansouri and Ricardo Campos | FALQU: Finding Answers to Legal Questions | ||||
| 10:30 | Coffee Break | ||||
| 11:00 | Keynote | Tjerk de Greef | Actionable content– how semantic data boosts legal professional search | ||
| 11:50 | Oral session | Aimen Louafi and Pauline Chavallard | Finding Unstructured References to French Collective Agreements in Legal Documents | ||
| Adam Wyner, Adeline Nazarenko, François Lévy and Haifa Zargayouna | Semantic Search in Legislation | ||||
| 12:30 | Lunch | ||||
| 13:30 | Poster session | Alexandre G. Lima, Jose G Moreno, Mohand Boughanem, Taoufiq Dkaki and Eduardo Aranha | Leveraging Positional Encoding to Improve Fact Identification in Legal Documents | ||
| Charles Courchaine and Ricky Sethi | Opening the TAR Black Box: Developing an Interpretable System for eDiscovery Using the Fuzzy ARTMAP Neural Network | ||||
| Kees van Noortwijk and Christian Hirche | Parsing User Queries using Context Free Grammars | ||||
| Maren Pielka, David Biesner, Rajkumar Ramamurthy, Tim Dilmaghani Khameneh, Bernd Kliem, Rüdiger Loitz and Rafet Sifa | Improving Automated Auditing Systems with Zero-Shot Text Matching and Sentence Transformers | ||||
| Masaharu Yoshioka, Juliano Rabelo, Randy Goebel, Yoshinobu Kano, Mi-Young Kim and Ken Satoh | Competition on Legal Information Extraction/Entailment (COLIEE) | ||||
| Nishchal Prasad, Mohand Boughanem and Taoufiq Dkaki | Exploring Semi-supervised Hierarchical Stacked Encoder for Legal Judgement Prediction | ||||
| Tobias Fink, Yasin Ghafourian, Georgios Peikos, Florina Piroi and Allan Hanbury | An Annotation Framework for Benchmark Creation in the Legal Case Retrieval Domain | ||||
| 15:00 | Coffee Break | ||||
| 15:30 | Keynote | Maura Grossman (together with ALTARS workshop) | The Limitations and Misuse of Information Retrieval in Legal Cases | ||
| 17:00 | End |
Invited speakers
Maura R. Grossman (Research Professor in the School of Computer Science at the University of Waterloo, an Adjunct Professor at Osgoode Hall Law School of York University, and an affiliate faculty member at the Vector Institute of Artificial Intelligence):
The Limitations and Misuse of Information Retrieval in Legal Cases
The authors recently undertook extensive validation testing on review technologies and processes to be employed in relation to a large, high-value/high-stakes legal case in Ireland. It was understood that legal discovery in these proceedings, with an estimated data set of over a quarter-billion documents, was not likely to be viable using commercially available electronic discovery software or conventional document review methods. Thus, as an alternative, a CAL® review platform, developed by Maura R. Grossman and Gordon V. Cormack was to be used to provide the active learning functionality underpinning the review. This CAL® system was designed to incorporate state-of-the-art developments in active learning as applied to legal information retrieval. Review practices, informed by the results of academic research into the limitations of human review, were to be used, both in parallel with and as an alternative to methods more typically employed in current legal practice. We report on the results of this research.
Milda Norkute (Lead Designer at Thomson Reuters Labs in Zug, Switzerland):
Evaluation of legal search from the end users’ perspective - current challenges and opportunities in industry
Legal research is ambiguous, challenging and time-consuming. This is because the “answers” to legal research questions are often not found in a single document and finding the answer can require putting together non-obvious and sometimes contradictory information from multiple documents and different sources. Therefore, designing and building search for legal professionals has unique challenges. This talk will focus on the question: how might we evaluate user’s search experience by using cross-disciplinary methods and inform the development of next generation of search for legal professionals? Existing practices to empathise with the users and understand their experience as well as questions that still remain unanswered will be discussed.
Tjerk de Greef (Director of Advanced & Search Technology in the global technology organization of Wolters Kluwer):
Actionable content– how semantic data boosts legal professional search
Today, Wolters Kluwer focusses on creating top-notch online services for a variety of professional customers worldwide. Everything we do is driven by state-of-the-art software, including Machine Learning based content enrichment microservices with the goal to enable advanced, complete, and semantic search experience for the legal professionals we support. A central pillar in these ‘expert solutions’ is actionable content, with the goal to transform and/or enriched existing (public) content sources. Actionable content allows customers to leverage the knowledge in these documents and align it in a way that is integrated into their daily work.
Such approaches require a pivotal thinking. The technology stack moves away from searching documents to true semantic search. In other words: the goal is to leverage data points that have a semantic meaning and created a linked graph. In this talk, we will elaborate on the NLP toolbox that enabling a better understanding of documents, including addressing endeavors in support of Legal Analytics and supporting asking question in natural legal language. We will also address our Machine Learning Life Cycle and toolbox to validate and measure search quality. Lastly, I will also address the approach Wolters Kluwer is following centralize around UX design patterns and link actionable content.
The Limitations and Misuse of Information Retrieval in Legal Cases
The authors recently undertook extensive validation testing on review technologies and processes to be employed in relation to a large, high-value/high-stakes legal case in Ireland. It was understood that legal discovery in these proceedings, with an estimated data set of over a quarter-billion documents, was not likely to be viable using commercially available electronic discovery software or conventional document review methods. Thus, as an alternative, a CAL® review platform, developed by Maura R. Grossman and Gordon V. Cormack was to be used to provide the active learning functionality underpinning the review. This CAL® system was designed to incorporate state-of-the-art developments in active learning as applied to legal information retrieval. Review practices, informed by the results of academic research into the limitations of human review, were to be used, both in parallel with and as an alternative to methods more typically employed in current legal practice. We report on the results of this research.
Milda Norkute (Lead Designer at Thomson Reuters Labs in Zug, Switzerland):
Evaluation of legal search from the end users’ perspective - current challenges and opportunities in industry
Legal research is ambiguous, challenging and time-consuming. This is because the “answers” to legal research questions are often not found in a single document and finding the answer can require putting together non-obvious and sometimes contradictory information from multiple documents and different sources. Therefore, designing and building search for legal professionals has unique challenges. This talk will focus on the question: how might we evaluate user’s search experience by using cross-disciplinary methods and inform the development of next generation of search for legal professionals? Existing practices to empathise with the users and understand their experience as well as questions that still remain unanswered will be discussed.
Tjerk de Greef (Director of Advanced & Search Technology in the global technology organization of Wolters Kluwer):
Actionable content– how semantic data boosts legal professional search
Today, Wolters Kluwer focusses on creating top-notch online services for a variety of professional customers worldwide. Everything we do is driven by state-of-the-art software, including Machine Learning based content enrichment microservices with the goal to enable advanced, complete, and semantic search experience for the legal professionals we support. A central pillar in these ‘expert solutions’ is actionable content, with the goal to transform and/or enriched existing (public) content sources. Actionable content allows customers to leverage the knowledge in these documents and align it in a way that is integrated into their daily work.
Such approaches require a pivotal thinking. The technology stack moves away from searching documents to true semantic search. In other words: the goal is to leverage data points that have a semantic meaning and created a linked graph. In this talk, we will elaborate on the NLP toolbox that enabling a better understanding of documents, including addressing endeavors in support of Legal Analytics and supporting asking question in natural legal language. We will also address our Machine Learning Life Cycle and toolbox to validate and measure search quality. Lastly, I will also address the approach Wolters Kluwer is following centralize around UX design patterns and link actionable content.