ICAConfPubs: A Dataset and User Interface for ICA Conference Papers (2003-2018)
Source: arXiv:2607.14234 · Published 2026-07-15 · By Hongtao Hao, Xinyue Chen, Jiye Sun, Yanling Zhao, Jing Zhang
TL;DR
This paper addresses the challenge of limited accessibility and usability of historical data from the International Communication Association (ICA) annual conferences, spanning from 2003 to 2018. The authors have compiled a comprehensive dataset including 27,466 papers, 21,038 authors, and 4,935 sessions, extracted primarily from the ICA's online conference programs. The dataset is made publicly available in multiple formats (CSV and JSON), along with an API and a user-friendly web interface to facilitate exploration and programmatic access. The work also includes semantic search functionality based on textual embeddings to improve retrieval of relevant papers beyond simple keyword matching. By analyzing yearly trends and applying topic modeling (using BERTopic) on the conference paper abstracts, the authors reveal the evolving thematic landscape of communication research over the fifteen-year period.
The resulting resource supports large-scale scientometric studies and offers new insights into the field’s development, scholarly collaboration, and topical diversity. The inclusion of an API and interface aims to lower the barrier for scholars and developers to utilize this rich dataset. The paper highlights key topic areas such as political communication, gender studies, digital media, health communication, and journalism. However, some limitations remain around incomplete author disambiguation and coverage gaps for years before 2003 and after 2018. Overall, this work fills a long-standing gap by making a crucial communications research dataset FAIR (findable, accessible, interoperable, reusable).
Key findings
- Dataset comprises 27,466 papers, 21,038 authors, and 4,935 sessions from ICA conferences 2003-2018.
- Number of ICA conference papers grew from 994 in 2003 to a peak of 2,348 in 2013 (+136%), then dropped sharply by 42.8% in 2014 due to data or program structure changes.
- Author records increased from 1,767 in 2003 to a peak of 4,424 in 2013, then dropped to 2,845 in 2014 but recovered to a record 5,201 by 2018, indicating increasing authorship activity and collaboration.
- Average authors per paper rose steadily from about 1.8 before 2014 to over 2.2 afterward, showing growing coauthorship trends.
- BERTopic identified 34 topics consolidated into 15 thematic categories, with Political Communication, Civic Engagement & Activism increasing from 9.8% of papers in 2003 to 18.8% in 2018.
- Semantic search using OpenAI’s text-embedding-3-small outperformed keyword searches for long natural language queries over titles and abstracts, improving relevant paper retrieval.
- The user interface supports browsing by paper, author, and session with filtering by year, enabling easier navigation of over 27K papers and 21K authors.
Methodology — deep read
Threat Model & Assumptions: The work does not focus on adversarial or bot-defense aspects; adversary modeling does not apply. This is a resource creation and data aggregation project aiming for open scientific access.
Data Collection & Processing: The authors scraped ICA's official conference data for years 2003-2018 from online program webpages hosted by allacademic.com. They used Selenium for dynamic scraping. The data includes paper metadata (title, abstract, year, session, division/unit), author details (name, affiliation, position), and session information (type, title, chair). Data from 2019 onward are excluded due to format challenges (only PDFs publicly available). The authors present three main datasets (papers, authors, sessions) released in CSV and JSON formats.
Data Characteristics & Preprocessing: Data shows changes in website structure between 2003-2004 (no sessions/divisions), 2005-2013 (full info), and 2014-2018 (interactive papers and changed schema). Author deduplication was not attempted, complicating exact author counts. Affiliation deduplication was also not done. Some papers lack author data.
Topic Modeling Method: The paper abstracts were processed through a BERTopic pipeline. This involved sentence transformer embeddings (all-mpnet-base-v2) over raw abstracts and lemmatized corpus for topic-word representations via spaCy. A grid search over UMAP and HDBSCAN parameters identified an optimal configuration yielding 34 topics with an initial 31.1% outlier rate reduced to 0.2% by outlier reassignment using cosine similarity threshold 0.3. Topics were manually consolidated into 15 categories by reviewing keywords and semantics.
Semantic Search: All paper titles and abstracts (27,466 papers) were embedded using OpenAI’s text-embedding-3-small model, storing embeddings in Pinecone vector database. User queries are embedded and compared via cosine similarity to enable semantic retrieval exceeding traditional keyword search.
User Interface and API: A React-based web application presents papers, authors, and sessions with linking and filtering features. An API in Next.js provides programmatic access. The interface displays details, session info, and author publications.
Evaluation: The paper mainly evaluates descriptive statistics on yearly trends (paper count, author count), topic trends over years (using figure 5), and outlier rate in topic models. Qualitative validation of topics and interface usability is described but no formal user study or external benchmarking.
Reproducibility: The paper provides dataset downloads in CSV/JSON, source code to scrape and process data, API documentation, and a live web app at https://ica.hongtaoh.com. The approach is transparent but dataset excludes years before 2003 and after 2018. No explicit seeds or hardware details mentioned for topic modeling training.
End-to-end Example: The authors scrape data for ICA 2013, extract papers and author info, embed abstracts with transformer models, cluster with HDBSCAN + UMAP, assign topics, then expose via their web UI and semantic search allowing exploration and new research insights.
Technical innovations
- Aggregation and public release of a large-scale, structured dataset of ICA conference metadata (papers, authors, sessions) spanning 16 years, not previously available in an accessible format.
- Application of BERTopic with a novel parameter grid search and outlier reassignment to produce coherent thematic categories for communication research abstracts.
- Integration of semantic search using OpenAI text embeddings combined with a vector database (Pinecone) for enhanced natural language querying across a large paper corpus.
- Development of an interactive React-based web interface combined with an API to facilitate user-friendly exploration and programmatic data analysis.
Datasets
- ICA Conference Papers (2003-2018) — 27,466 papers — scraped from ICA annual conference online programs
- ICA Authors Metadata — 21,038 authors — same source
- ICA Sessions Metadata — 4,935 sessions — same source
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.14234.
Fig 1: The interface of ICAConfPubs live website and paper details. The headers include papers, authors,
Fig 2: Past ICA annual conferences.
Fig 3: Online program example of ICA annual conference in 2003.
Fig 5: Distribution and temporal trends of 15 BERTopic-identified thematic categories across ICA
Fig 6: Screenshots of the Papers page in the user interface
Fig 8: Author Page
Fig 9: Session Page
Limitations
- Data only covers ICA conferences from 2003 to 2018; newer years (2019 onward) are missing due to PDF-only availability.
- Author name disambiguation was not performed, so authors with identical names may be conflated, affecting author-level analyses.
- Affiliation deduplication was not done, complicating affiliation-based insights.
- 2014 conference program changes produce a sharp data discontinuity in paper and author counts, complicating temporal trend interpretation.
- No formal user study or external benchmark conducted to evaluate the semantic search UI or topic model coherence quantitatively.
- The dataset excludes submissions and presentations prior to 2003, limiting longitudinal analyses.
Open questions / follow-ons
- How can author name disambiguation be effectively applied to the ICA dataset to improve author-level analyses?
- Can the semantic search approach be quantitatively benchmarked against traditional keyword search and user queries for relevance?
- How do the uncovered topic trends correlate with broader shifts in communication research from journals and other conferences?
- What methodologies can be used to integrate ICA conference data with journal publication data for deeper scientometric studies?
Why it matters for bot defense
While this paper does not address bot detection or CAPTCHA systems directly, it presents best practices for building and releasing large structured datasets with APIs and user interfaces that support semantic search. Bot-defense practitioners can learn from the authors' approach to data aggregation, semantic embedding, and retrieval techniques, which can be adapted to analyze textual logs or metadata in security contexts. The use of transformer embeddings combined with a vector database for semantic similarity retrieval is a technical pattern applicable in CAPTCHA user behavior analysis or malicious content detection. Also, this work exemplifies how to create FAIR data ecosystems that facilitate large-scale empirical analyses, a principle underpinning robust threat intelligence sharing.
Cite
@article{arxiv2607_14234,
title={ ICAConfPubs: A Dataset and User Interface for ICA Conference Papers (2003-2018) },
author={ Hongtao Hao and Xinyue Chen and Jiye Sun and Yanling Zhao and Jing Zhang },
journal={arXiv preprint arXiv:2607.14234},
year={ 2026 },
url={https://arxiv.org/abs/2607.14234}
}