Abstract: This project seeks to shed light on the intricacies and obscurity of online Terms & Conditions (T&Cs) that users frequently agree to without comprehensive comprehension. We employed ChatGPT (LLM) and Wolfram Language (WL) to conduct a shallow semantic examination of 161 T&Cs documents from 70 different companies, and established a web service for on-demand T&Cs analysis. We discovered that while LLM’s capabilities are hampered by token limits and the quality of prompts, it holds tremendous promise for tackling intricate text-processing tasks that demand extensive domain expertise. We plan to further harness this capability, fostering a community-driven movement reminiscent of the open-source revolution.
Background
Background
“I have read and agree to the Terms” is the biggest lie on the web. - https://tosdr.org/
The Problem
The Problem
The prevalent issue with online Terms & Conditions (T&Cs) is akin to a complex novel - they are far too long and intricate, often resulting in a ‘Too Long; Didn’t Read’ (TL;DR) scenario. Users are overwhelmed by the sheer volume and intricacy of these T&Cs, which leads to a sense of apathy and ‘redundant’ consent. This situation mirrors the early days of the open-source movement when developers frequently re-implemented the same tools due to a lack of a collaborative platform for code reuse, resulting in an enormous waste of effort.On the other side of the coin, companies grapple with a cost-risk dilemma when formulating these T&Cs. Similar to buying insurance, companies face a high cost of drafting comprehensive T&Cs to protect themselves from possible future legal disputes. They have to balance the legal expenses with the risk of potential attacks, leading to convoluted T&Cs to cover as many eventualities as possible. The result is a tough environment where individuals and corporations alike are entangled in a dense web of complex T&Cs, neither fully understanding nor benefiting from the current setup.
The Hypothesis
The Hypothesis
We hypothesize that atomizing T&Cs could be a transformative approach to mitigate the current issues, much like the modularity in the open-source movement. Breaking down T&Cs into more digestible, standard units could bring about various benefits. First, it becomes easier to standardize these atomic terms, creating a common language for users and businesses alike. This also paves the way for a GitHub-like repository of T&Cs, where standardized atomic terms could be easily accessed, shared, and updated by the community.
For users, this atomization means they can ‘inherit’ their consent for standard terms across different services, eliminating the redundancy of reading and consenting to similar terms repeatedly. For companies, especially start-ups, this modular approach reduces the legal overhead of crafting T&Cs from scratch. They can compose their T&Cs by reusing standardized terms, much like assembling software from open-source libraries.
On the other hand, large corporations might face tighter regulations and scrutiny when introducing complex, non-standard terms. However, this could be a boon in disguise, encouraging them to be transparent and fair in their T&Cs, and thus fostering greater trust from users. In essence, our hypothesis envisions a mutually beneficial ecosystem, where T&Cs become accessible, comprehensible, and user-friendly, while companies can ensure their legal safety in a cost-effective manner.
For users, this atomization means they can ‘inherit’ their consent for standard terms across different services, eliminating the redundancy of reading and consenting to similar terms repeatedly. For companies, especially start-ups, this modular approach reduces the legal overhead of crafting T&Cs from scratch. They can compose their T&Cs by reusing standardized terms, much like assembling software from open-source libraries.
On the other hand, large corporations might face tighter regulations and scrutiny when introducing complex, non-standard terms. However, this could be a boon in disguise, encouraging them to be transparent and fair in their T&Cs, and thus fostering greater trust from users. In essence, our hypothesis envisions a mutually beneficial ecosystem, where T&Cs become accessible, comprehensible, and user-friendly, while companies can ensure their legal safety in a cost-effective manner.
Work Summary
Work Summary
Challenges
Challenges
◼
In-depth legal domain knowledge requirement: The project faced initial difficulties due to the sophisticated language and context used in T&Cs. Understanding and processing these documents require specific legal expertise.
◼
Labor-intensive text processing: Reviewing, normalizing, and promoting terms from a large volume of text presented a considerable challenge.
Approach
Approach
To counter these challenges, the project adopted a two-fold approach.
◼
Utilization of Large Language Models (LLMs): Deployed LLMs to handle the complexity of legal text, bridging the knowledge gap and cutting labor costs.
◼
Embracing Community Collaboration: Built a web based T&C service to leverage the power of community collaboration.
Work Performed
Work Performed
1
.Conducting a Meta analysis across 70 companies and 161 files using chatGPT
1
.1
.T&Cs data exploration
1
.1
.1
.WordCloud to reveal the naming discrepancies
1
.1
.2
.Histogram on token count needed to process the files
1
.2
.Text processing and summarization
1
.2
.1
.chatGPT assisted prompt engineering
1
.2
.2
.chatGPT prompts for topic-clause extraction
1
.2
.3
.Summarization comparison between GPT-3.5 & GPT-4
1
.3
.T&Cs Visualizations
1
.3
.1
.NearestNeighborGraph
1
.3
.2
.FeatureSpacePlot
2
.Deploying a cloud-based T&C service
2
.1
.URL content evaluator
2
.2
.Splitting documents for summary generation from chatGPT
2
.3
.Testing chatGPT with no stopwords
2
.4
.Deploy to Cloud
3
.Conclusions and learnings
Meta analysis on T&Cs
Meta analysis on T&Cs
T&Cs data evaluation
T&Cs data evaluation
Text processing and summarization
Text processing and summarization
T&Cs Visualizations
T&Cs Visualizations
To unravel the complex web of Terms and Conditions documents, I employed a multi-faceted approach. Here are the key steps:
Extracting Summaries with GPT: The GPT model was used to distill summaries from each of the T&Cs files, essentially reducing lengthy, intricate text into more manageable content.
Generating Sentence-Level Embeddings: The GPT model was then used to derive sentence-level embeddings for these summaries across various services. Each embedding had a dimension of 1536.
However, this approach wasn’t without limitations. The use of GPT-3.5 for extracting summaries posed some challenges, such as inconsistencies in the output format and persisting references to specific company or service names. Despite these issues, the method yielded intriguing insights.
Extracting Summaries with GPT: The GPT model was used to distill summaries from each of the T&Cs files, essentially reducing lengthy, intricate text into more manageable content.
Generating Sentence-Level Embeddings: The GPT model was then used to derive sentence-level embeddings for these summaries across various services. Each embedding had a dimension of 1536.
However, this approach wasn’t without limitations. The use of GPT-3.5 for extracting summaries posed some challenges, such as inconsistencies in the output format and persisting references to specific company or service names. Despite these issues, the method yielded intriguing insights.
NearestNeighborGraph
NearestNeighborGraph
Here, we used two nearest neighbors for plotting, with different colors representing sentences from different services.
This visualization revealed four distinct clusters, suggesting correlations within each cluster. Particularly noteworthy was the left bottom cluster, where summaries from different companies displayed similar semantic meaning.
This alignment suggests the potential for standardizing particular terms across multiple companies, lending credibility to our initial hypothesis.
This visualization revealed four distinct clusters, suggesting correlations within each cluster. Particularly noteworthy was the left bottom cluster, where summaries from different companies displayed similar semantic meaning.
This alignment suggests the potential for standardizing particular terms across multiple companies, lending credibility to our initial hypothesis.
vectorCompanies=Association[Catenate@Values@MapIndexed[#1->First[#2]&,termAsso,{2}]];g=NearestNeighborGraph[Values[Merge[termAsso,Total]],2,VertexCoordinates->Automatic];Graph[g,VertexLabels->Thread[VertexList[g]->(Placed[#,Tooltip]&/@Lookup[AssociationMap[Reverse,Merge[termAsso,Total]],VertexList[g]])],VertexStyle->Normal[ColorData[3]/@vectorCompanies],VertexSize->5]
Out[]=
23679200
FeatureSpacePlot
FeatureSpacePlot
The FeatureSpacePlot offered even more intriguing observations.
Using TSNE for Dimension Reduction: Here, we employed the t-Distributed Stochastic Neighbor Embedding (TSNE) method to identify clusters in a high-dimensional space and subsequently perform dimension reduction.
Isolating Correlated Sentences: When used in conjunction with the FeatureSpacePlot, TSNE amplified the isolation of potential clusters, thus making correlated sentences from different services more distinguishable. These isolated clusters highlight the potential commonalities among various terms across different companies, further strengthening our case for T&Cs standardization.
Using TSNE for Dimension Reduction: Here, we employed the t-Distributed Stochastic Neighbor Embedding (TSNE) method to identify clusters in a high-dimensional space and subsequently perform dimension reduction.
Isolating Correlated Sentences: When used in conjunction with the FeatureSpacePlot, TSNE amplified the isolation of potential clusters, thus making correlated sentences from different services more distinguishable. These isolated clusters highlight the potential commonalities among various terms across different companies, further strengthening our case for T&Cs standardization.
dr=DimensionReduction[Values[Merge[termAsso,Total]],2,Method->"TSNE"]
FeatureSpacePlot[Merge[termAsso,Total],FeatureExtractor->dr]
Cloud-based T&Cs service
Cloud-based T&Cs service
URL content evaluator
URL content evaluator
Split File when longer than 4098 tokens
Split File when longer than 4098 tokens
Deploy to Cloud
Deploy to Cloud
Conclusions and learnings
Conclusions and learnings
What is new? -> now we have LLM superpower
What is new? -> now we have LLM superpower
Reflecting on this project, several key insights have emerged:
1. Thoughts on GPT Models
GPT models, particularly GPT-4, have demonstrated their immense potential in tasks like text processing and summarization. When compared to human labor, the cost efficiency of these models is truly impressive. However, there are limitations that need to be acknowledged. One notable constraint is the token limit, which, due to technical restrictions, poses a challenge in processing large amounts of data.
2. Text Processing Tips with GPT Models
GPT-4 has shown significantly improved results compared to GPT-3 in extracting topics and summaries using zero-shot learning. The consistency in output format with GPT-4 is remarkable and outperforms GPT-3 in this regard. However, the choice between one-shot learning with GPT-3 and zero-shot learning with GPT-4 may vary case by case considering cost-benefit analysis. Furthermore, we learned that GPT can process texts even without stopwords, potentially reducing costs. However, stemming, which might increase the token count, should be cautiously used as it might not necessarily improve the performance and could inflate the costs.
3. Choosing LLM for Summarization vs. Local Clustering with Embeddings
When deciding between using an LLM like GPT for summarization or extracting embeddings for local clustering, cost and reliability considerations come into play. With its incredible processing power and speed, GPT models provide cost-effective and reliable solutions. On the other hand, local clustering demands significant computational resources and might still require considerable effort to evaluate the results. Thus, depending on the specific needs of the task, the choice between the two could be informed by these considerations.
In conclusion, this project has provided invaluable learning experiences in using Large Language Models for text processing and summarization tasks. The insights gained here will be instrumental in further leveraging these models in similar domains.
1. Thoughts on GPT Models
GPT models, particularly GPT-4, have demonstrated their immense potential in tasks like text processing and summarization. When compared to human labor, the cost efficiency of these models is truly impressive. However, there are limitations that need to be acknowledged. One notable constraint is the token limit, which, due to technical restrictions, poses a challenge in processing large amounts of data.
2. Text Processing Tips with GPT Models
GPT-4 has shown significantly improved results compared to GPT-3 in extracting topics and summaries using zero-shot learning. The consistency in output format with GPT-4 is remarkable and outperforms GPT-3 in this regard. However, the choice between one-shot learning with GPT-3 and zero-shot learning with GPT-4 may vary case by case considering cost-benefit analysis. Furthermore, we learned that GPT can process texts even without stopwords, potentially reducing costs. However, stemming, which might increase the token count, should be cautiously used as it might not necessarily improve the performance and could inflate the costs.
3. Choosing LLM for Summarization vs. Local Clustering with Embeddings
When deciding between using an LLM like GPT for summarization or extracting embeddings for local clustering, cost and reliability considerations come into play. With its incredible processing power and speed, GPT models provide cost-effective and reliable solutions. On the other hand, local clustering demands significant computational resources and might still require considerable effort to evaluate the results. Thus, depending on the specific needs of the task, the choice between the two could be informed by these considerations.
In conclusion, this project has provided invaluable learning experiences in using Large Language Models for text processing and summarization tasks. The insights gained here will be instrumental in further leveraging these models in similar domains.
Acknowledgment
Acknowledgment
Success is always a collaborative effort and this project is no exception. I would like to extend my deepest gratitude to my mentors who made this achievement possible:
Christopher Wolfram: Your depth of knowledge in Wolfram language and your knack for rapidly prototyping viable solutions left me in awe. Your guidance has been pivotal in shaping this project.
Maria Sargsyan: Your invaluable insights and supportive presence have truly made a difference. Whether it was bouncing off ideas or navigating through technical roadblocks, your help has been instrumental in reaching this stage.
Daniel George: A special thank you for your insightful feedback that helped hone the focus of this project. Your recognition and encouragement served as a great motivator.
Christian Pasquel, Fez Zaman and many more.
Christopher Wolfram: Your depth of knowledge in Wolfram language and your knack for rapidly prototyping viable solutions left me in awe. Your guidance has been pivotal in shaping this project.
Maria Sargsyan: Your invaluable insights and supportive presence have truly made a difference. Whether it was bouncing off ideas or navigating through technical roadblocks, your help has been instrumental in reaching this stage.
Daniel George: A special thank you for your insightful feedback that helped hone the focus of this project. Your recognition and encouragement served as a great motivator.
Christian Pasquel, Fez Zaman and many more.
References
References
CITE THIS NOTEBOOK
CITE THIS NOTEBOOK
Unveiling the veil: a semantic analysis and visualization of web terms & conditions
by Dongming Jin
Wolfram Community, STAFF PICKS, July 12, 2023
https://community.wolfram.com/groups/-/m/t/2959906
by Dongming Jin
Wolfram Community, STAFF PICKS, July 12, 2023
https://community.wolfram.com/groups/-/m/t/2959906