Contextual Enrichment of Crowds from Mobile Phone Data through Multimodal Geo-Social Media Analysis
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThis paper reports a method to enrich mobile phone data with social media data. The method was validated via a case study.
In general, the motivation and method are interesting and potentially useful. I have several suggestions for the authors to consider.
1) The linking of social media data with crowds extracted from mobile phone data is the key. In the paper, this is simply done by checking the spatial and temporal information. How do you know that the social media data are indeed related to the crowds, but not something else? For analyzing social media data, it might make sense to apply a similar methodology like the one with baselines for the crowd detection from mobile phone data.
2) The poor location accuracies of mobile phone data and social media data indeed post significant challenges related to the usefulness of the results, e.g., for crowd management. From your perspective, what would be needed or what quality levels should be in order to make the results useful for crowd management and other relevant applications.
3) I would recommend briefly describing the JSTTS-GeoGSOM model in Section 3.2. You should at least give a high-level description to introduce what it can do, and methods/algorithms behind.
4) Section 3.2, use of LLM. Did you manually check the LLM output? If not, how did you ensure its output quality?
5) Section 4.2.1.: How comes that the percentage of 65+ visitors is so low? How is a visitor being defined and identified?
6) Section 4.2.1: It is still unclear to me how you match the two mobile phone datasets.
Author Response
First of all, we would like to thank you for the positive evaluation and helpful comments on our paper. This has enabled us to further improve the quality of our submission.
In the attached document, we will refer to your comments and explain and justify our changes to the manuscript.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsI think that the approach itself is quite interesting—combining large-scale data on presence of people with smaller-scale data on sentiment of people or context of presence in order to get the best out of both datasets. That said, I do find some issues with the framing and content of the analysis—some big, some very small, all addressable with some revision—which I’ll list below:
1. I'm not sure I buy the spin that this is useful for real-time crowd management or intervention. Perhaps you more meant this analysis could be useful for understanding past events in order to prevent future events as opposed to real time intervention/disaster prevention. If the former is true, I think you should perhaps just add some more specificity to the language you use to make sure that's clear to the reader. However, if it's the latter, I think there are two major issues with this: 1) the problems you encounter with data sparsity and geographic non-specificity in the geo-social media data (the "Vienna" bounding box issue) is likely to be an issue in any context or surrounding any major event, and increasingly so as people become more concerned about data privacy and opt out of detailed location sharing more often. I appreciate that you present the given case study as a demonstration of methodology despite the current dataset's limitations, but I struggle to think of situations in which it could be applied without those limitations. 2) I haven't ever heard of anyone having access to real-time CDR data --- in my experience, CDR data needs to be preprocessed and go through cleaning, anonymisation, and sometimes even synthesization or the addition of noise before, eg, researchers can deal with it. Maybe I'm projecting my own limited experience here, or maybe police or government organizations have access to this kind of real-time cell phone data? If so, lay that context out more specifically --- but if not, I can't see this being useful for gaining any kind of real-time understanding of a disaster situation or doing live crowd management. That said, I think this is an introduction/conclusion issue more than a substantive issue --- I *do* think there are very interesting use cases for this, from my perspective they just surround post-event analysis (and, perhaps, due to the sparsity of geo-social media data, "events" being longer-term, several-week - several-month events like extended protests, long festivals, or pandemic situations as opposed to single-day events). I would love to see you reframe a bit the introduction and discussion, as they currently lean quite heavily on the "real-time crowd management" perspective, and instead frame the research in the context of use-cases that are more grounded in the limitations of the dataset.
2. Small typos: line 78 – “anonymous calling patters”; line 95 “social media post → social media posts”
3. Age distribution – did you have age distribution of the entire population of users in the dataset, or specific age distributions describing the visitors to each individual grid cell? If the former, I’m not sure how accurately this age distribution will reflect the age distribution of actual visitors to the pride festival (which I would imagine skew younger in this case; and outside of this specific case I would say most urban events are somewhat age-specific) and I would honestly omit this part of the analysis; if the latter, this is a detail that is a significant ask of CDR datasets and often isn’t attached and so it might be best to frame it as an optional detail as opposed to a key part of the methodology itself (in this case, may be useful to show the sensitivity of results to this walking speed to show that the method doesn’t rely on detailed knowledge of age distributions).
4. I think assigning all tweets from the “Vienna” bounding box to random points in the city is quite crude and may just be adding noise to the analysis/obscuring genuine trends. Again, I respect that this is just a demonstration of methodology, but it is an issue that I believe most researchers will similarly run into when implementing the methodology and so it would be helpful to present a way to deal with it in a way that doesn’t obscure the analysis. Similarly, is there a more sophisticated way that you can impute the missing values in the mobile phone dataset, as opposed to assigning all 2s? Perhaps estimating based on surrounding grid cells/usual activity in that location?
5. Do Twitter and Facebook add some bit of random noise to locations for privacy reasons, even when they are specific points? I know that Twitter has at least done this in past periods, not sure about the period of your case study — it would just be worth to note if there is noise, and how much.
6. Why do you extract specifically posts with the pride theme, as opposed to all tweets in the Pride festival’s location? I actually find this to be quite impactful — for one, I believe that the words you selected would in general skew sentiment in a positive direction (for example, people angrily complaining about being held up by the pride parade while trying to walk elsewhere or something like that or people who are a threat to the pride parade/the LGBTQ community may describe it using different words or may not mention the cause of the parade at all). This is especially true for the step where you limit clusters to those including “pride” — this is a very positive, empowering word and skews the “contextual enrichment” you’re going to get in a very specific direction. For example, in line 364, you state “These tables show that the semantic content…was about the Vienna Pride Parade” — analysis like this isn’t very meaningful as you filtered for this to be true. Given the use case presented, I think extracting posts with these specific words (1) defeats some of the purpose of truly understanding crowd sentiment or potential emergency/unrest and (2) severely limits your dataset.
7. In Table 3, if I’m interpreting the “post_ids” column correctly, it seems many crowds only have one associated post — is this true? If so, it may raise eyebrows — it is a difficult sell to assign the sentiment of one tweet to 19,000+ people. As briefly discussed in the comment above, this could be helped by expanding your dataset beyond just tweets or posts containing specific words. Alternatively, are there other sources of semantic data that could be incorporated?
8. The scaling beginning on line 323 could be laid out a bit more clearly (did you match each spatial unit in dataset A to the spatial unit it most largely overlapped in dataset B? Or partial out visitors proportionally to overlap size?) It could also potentially be moved to the main methodology section as i would imagine this type of issue might come up semi-frequently in practice (optional, of course).
9. I find the Discussion section to be quite disjointed, and it reads more as a list of limitations than a meaningful discussion. Perhaps this could be tied together more smoothly.
10. The last paragraph of the literature review could be clearer and more straightforward in terms of what specifically you are bringing to the bodies of work described.
Author Response
First of all, we would like to thank you for the positive evaluation and helpful comments on our paper. This has enabled us to further improve the quality of our submission.
In the attached document, we will refer to your comments and explain and justify our changes to the manuscript.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsThe paper was interesting to read and topic is relevant with the current scientific merit. Nevertheless, privacy protection measures might be considered beyond the GDPR, since the authors are working closely with sensitive information. I would recommend referring to existing literature on the potential risks that might be occur against personal location privacy. This could be then reflected on the future research directions.
Comments on the Quality of English LanguageThe quality of language was good. The text should be proofread for typos.
Author Response
First of all, we would like to thank you for the positive evaluation and helpful comments on our paper. This has enabled us to further improve the quality of our submission.
In the following, we will refer to your comments and explain and justify our changes to the manuscript. You will also find the updated PDF attached, as well as one file that details the changes.
Author Response File: Author Response.pdf
Reviewer 4 Report
Comments and Suggestions for AuthorsThis study achieves contextual enrichment of crowd data by integrating mobile phone data with geo-social media. This is an intriguing piece of research that could be applied in future studies on population activity and emergency event management.
My major concern is the definition of 'crowd' in this study. A grid cell is defined as a 'crowd' when its positive z-score falls within the 90% confidence interval. This definition is problematic because, for instance, if grid cell A has a crowd size of 10, and its neighboring cells have crowds smaller than 10, grid cell A may be classified as a 'crowd cell.' On the other hand, if grid cell B has a crowd size of 100, and its neighboring cells have crowds between 60 and 80, grid cell B may also be classified as a 'crowd cell,' but its neighboring cells may not be classified as such. This creates an inconsistency, as both A (with a crowd of 10) and B (with a crowd of 100) are considered 'crowd cells,' while the neighboring cells of B (with crowds between 60 and 80) are not.
Author Response
First of all, we would like to thank you for the positive evaluation and helpful comments on our paper. This has enabled us to further improve the quality of our submission.
In the attached document, we will refer to your comments and explain and justify our changes to the manuscript.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
Comments and Suggestions for AuthorsThank you for your thoughtful response to my comments --- you've both corrected some misinterpretations that I had and addressed some issues with framing and clarity within the paper.
