Assessing Perceived Landscape Change from Opportunistic Spatiotemporal Occurrence Data

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Review of Land 3105013

Assessing perceived landscape change from opportunistic spatio-temporal occurrence data

 

Overall:

Very nice paper; it addresses an important issue in landscape perception dealing with change, and does it from a relatively novel perspective and approach. In particular, the Social-Ecological-Technological System (SETS) framework is effectively employed in characterizing and comparing results across the five case studies, and the discussion provides a good comparison of the relative strengths and weaknesses of the different datasets used. The paper will make a useful contribution to the literature and fits nicely with the theme of the special issue.

 

Intro:

Good, but see L48 under minor issues below

 

Lit Review:

1.      Nice discussion of Hull and McCarthy (1988)! I’ve always felt this was an important paper and it’s good to see you cite it.

2.      The review as whole is very helpful, especially the methodological issues regarding bias/ representativeness and how best to characterize them.

 

Materials and Methods

1.      This section could benefit from descriptively labeled subsections to better guide the reader (e.g., 3.1. Framework for Analysis)

2.      While the supplementary materials provide excellent documentation of your approach for readers who would like the full details of your approach, the text in the paper for each of the case studies should at least provide a sketch of your analytical approach. As it now reads, this level of detail is uneven across the studies. While I recognize some datasets may be more complex and thus require more information, for some studies e.g. case study three, there’s little or no detail of how the data were analyzed

 

Results and Discussion

1.      Combining these two sections seems appropriate but lessens the ability to compare findings across case studies. This, however, is adequately addressed in the Conclusions, which is longer than typical papers but serves a number of important purposes.

2.      Case study 1 draws from previous master’s research by Tautenhahn, but it’s not clear to what extent this treatment is summarizing that work or providing a new analysis. While the work is credited, the extent of contribution of that work should be more clearly described.

 

Minor issues:
L21 delete “already”

L43 clarify or delete “algorithms” – what kinds of algorithms influence landscape behavior/engagement?

L48 clarify or reword “these three components”; looking back at the first paragraph it is unclear which three components this refers to.

L83 You introduce the VGI acronym in the abstract but you should also spell out the full term again here in the text of the paper.

L113 Delete “(ibid)”; not needed.

L154 The SETS acronym was already spelled out in L49

L230 revise “As a second example” to read “In the second study”

L235 typo last word “receiv”

L228 for some of the case studies you mention the size of the dataset/sample of photos but don’t do it for others—I would be consistent and include that info in the text for each study.

L288 “the author’s” is unclear; does it refer to Tautenhahn or Dunkel and Burghardt?

Fig 3 Axes should be labeled

L492 “to underpin this approach” could be more clearly worded e.g., “to illustrate this utility”

L562 (Methods and Data) should read (Materials and Methods)

Comments on the Quality of English Language

English is fine

Author Response

Comments 1: 
> Very nice paper; it addresses an important issue in landscape
perception dealing with change, and does it from a relatively
novel perspective and approach. In particular, the Social-
Ecological-Technological System (SETS) framework is effectively
employed in characterizing and comparing results across the five
case studies, and the discussion provides a good comparison of
the relative strengths and weaknesses of the different datasets
used. The paper will make a useful contribution to the literature
and fits nicely with the theme of the special issue.
Intro:
> Good, but see L48 under minor issues below

> Lit Review:
> 1.
> Nice discussion of Hull and McCarthy (1988)! I’ve always felt this
was an important paper and it’s good to see you cite it.
> 2.
> The review as whole is very helpful, especially the
methodological issues regarding bias/ representativeness and
how best to characterize them.

Response 1: Thank you for these very supportive suggestions and helpful comments. 

Comments 2: 
> Materials and Methods
> 1.
> This section could benefit from descriptively labeled subsections
> to better guide the reader (e.g., 3.1. Framework for Analysis)

Response 2: Agreed. We added three subsections to better structure Materials and Methods:
3.1 Framework for analysis; 3.2 Data Collection and Preprocessing; 3.3 Signed Chi Equation

Comments 3: 
> 2.
> While the supplementary materials provide excellent
> documentation of your approach for readers who would like the
> full details of your approach, the text in the paper for each of the
> case studies should at least provide a sketch of your analytical
> approach. As it now reads, this level of detail is uneven across
> the studies. While I recognize some datasets may be more
> complex and thus require more information, for some studies e.g.
> case study three, there’s little or no detail of how the data were
> analyzed

Response 3: Agreed, thank you for pointing this out. We made several changes to better account
for data analysis steps. We added several sentences to better describe data collection (L223-230), 
preprocessing (LL243-247). Specifically, we also added a better description of these steps 
for case study three (LL468-478). We have also added three appendices (A1-A3) that provide
readers with the full results for case studies, beyond what we were able to discuss directly in the article. 
These should help to better understand the research design and to draw conclusions or 
transfer knowledge beyond what could be discussed in this article. Lastly, we added
a Jupyter Notebook for case study 3 that was missing in our previous submission. This is
now referenced as Supplementary Materials 01 in the paper. 

The data that we collected and use to analyze results and figures are now
available in a separate data publication and cited accordingly:
Dunkel, A.; Burghardt, D. Supplementary materials for the publication "Assessing Perceived 
Landscape Change from Opportunistic Spatio-Temporal Occurrence Data". Opara digital research 
archive. DOI: 10.25532/OPARA-572.

Comments 4: 
> Results and Discussion
> 1.
> Combining these two sections seems appropriate but lessens
> the ability to compare findings across case studies. This,
> however, is adequately addressed in the Conclusions, which is
> longer than typical papers but serves a number of important
> purposes.

Response 4: Thank you for pointing this out. We want to mention that we 
significantly shortened conclusions, to better emphasize findings, in response
to reviewer 3.

Comments 5: 
> 2.
> Case study 1 draws from previous master’s research by
> Tautenhahn, but it’s not clear to what extent this treatment is
> summarizing that work or providing a new analysis. While the
> work is credited, the extent of contribution of that work should be
> more clearly described

Response 5: Indeed this was not as clearly formulated. We have updated
this section to clearly support dustinguishing contributions. In particular,
we want to highlight that Claudia Tautenhahn was updated about the progress
of this manuscript and agreed to our updated formualation of this section.
For convenience, we copy updated LL309-318 here:

"Data collection and the analysis for this study presented relatively few challenges. 
Claudia Tautenhahn contributed a list of potentially affected places, based on a 
priori knowledge that she gained from literature and field observations. Because 
Instagram enables place-based communication through a named gazetteer of user-contributed 
places, these places could be used to directly query and filter data. For the 13 
given lo-cations and 40 assigned Instagram places provided by Tautenhahn, we retrieved 
all posts, starting in 2019 and going backward in time. To emphasize, the data collection 
was done by the authors of this article. The visuals presented here were generated 
independently of the master's thesis. Tautenhahn's thesis is based on the same data 
and includes additional qualitative surveys and interpretations, which we cite accordingly." 

Comments 6: 

> Minor issues:
> L21 delete “already”
> L43 clarify or delete “algorithms” – what kinds of algorithms
> influence landscape behavior/engagement?
> L48 clarify or reword “these three components”; looking back at
> the first paragraph it is unclear which three components this
> refers to.
> L83 You introduce the VGI acronym in the abstract but you
> should also spell out the full term again here in the text of the
> paper.
> L113 Delete “(ibid)”; not needed.
> L154 The SETS acronym was already spelled out in L49
> L230 revise “As a second example” to read “In the second study”
> L235 typo last word “receiv”
> L228 for some of the case studies you mention the size of the
> dataset/sample of photos but don’t do it for others—I would be
> consistent and include that info in the text for each study.
> L288 “the author’s” is unclear; does it refer to Tautenhahn or
> Dunkel and Burghardt?
> Fig 3 Axes should be labeled
> L492 “to underpin this approach” could be more clearly worded
> e.g., “to illustrate this utility”
> L562 (Methods and Data) should read (Materials and Methods)

Response 6: Thank you very much for spotting these additional errors and typos.
We corrected all errors and followed your suggestions.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

In this manuscript the authors describe various ways to use VGI data to better understand perceptions of landscapes by human users of these landscapes. I found the synthesis interesting and informative. I have a number of specific concerns below, but my broad overall concerns are 1) at times the manuscript drifts to broadly (e.g., multiple references to using VGI for biodiversity monitoring, a field that includes a diversity of work addressing the challenges with imperfect data (e.g., the whole field of species distribution modeling) or when comparing different sources of bias in different parts of the manuscript and 2) when the results leans in too hard on quantitative interpretations of imperfect data (e.g., my concern about the inference on twitter below).  

On the first point above, I would suggest removing most discussions of biodiversity monitoring.

Next, it is not clear how the merits of the five case studies are evaluated. Was it on the SETS framework or the use of multiple different VGI sources to reduce bias. Both are interesting, but the lack of clarity perhaps this suggests that the takehome message of this paper is not clear. They could both be in the paper but this needs to be better framed and presented.

One last thing, I can appreciate the note that the full data acquisition is beyond the scope of this paper, but to my knowledge there is no public API for Instagram. When I looked at the python code, it seemed it was pulling in existing data rather then using the API. This is not a problem, but could the methods make clear what was existing data as compared to novel pull from an API. I suppose this also feeds back to a concern about the time window of the datasets. Since it is 2024, are the same APIs available to replicate the suggested workfellows or solutions in the manuscript?

 

Specific comments

Line 30 – since Land uses a numbered citation format could you not the year of this piece of work? (and at line 62)

Line 33 – Perhaps in addition to human and phenomenal, the authors could add environmental? Or expand on why pheromonal is the best term here?

The transition between line 42 and 43 could be expanded

In line 135 the authors state “better reflect the user's own value system,” what is this better than?

In line 150 the authors state “…categorizing three 150 broad umbrella biases found in opportunistic data: Ecological, Social, and Technological.” How do these source of bias differ or relate to the sources of bias noted above (line 106) and the two proposed solutions from the literature?

 

The authors state at line 427 “The first two case studies showed relatively weak ecological coupling (E),” However one could argue in Norway and in the national parks there is a clear seasonal variation.

I am not sure I an see in figure four “The rise and fall of Twitter”. Particularly given the fact that the data ends in 2017

Author Response

Comments 1: 
> In this manuscript the authors describe various ways to use VGI
data to better understand perceptions of landscapes by human
users of these landscapes. I found the synthesis interesting and
informative. 

Response 1: Thank you for these suggestions and helpful comments. 

Comments 2: 

> I have a number of specific concerns below, but my
> broad overall concerns are 1) at times the manuscript drifts to
> broadly (e.g., multiple references to using VGI for biodiversity
> monitoring, a field that includes a diversity of work addressing
> the challenges with imperfect data (e.g., the whole field of
> species distribution modeling) or when comparing different
> sources of bias in different parts of the manuscript and 2) when
> the results leans in too hard on quantitative interpretations of
> imperfect data (e.g., my concern about the inference on twitter
> below).
> On the first point above, I would suggest removing most
> discussions of biodiversity monitoring.

Response 2: 

We have carefully revised the literature review and the paper throughout, including 
the conclusions, to make clearer our motivations for referencing biodiversity monitoring 
literature. Specifically, biodiversity monitoring literature is critical to understanding 
and analyzing aspects of the ecological domain through the lens of social media, 
which we do in (e.g.) case studies 4.3 and 4.5. The ecological domain is one of the 
three poles used in our work and is a critical aspect in the analysis of landscape 
perception. Our method of using umbrella communities to disentangle social and ecological 
patterns (case study 4.5) draws directly from biodiversity monitoring research (Rapacciuolo et al. 
2021), so we feel the need to cite this related area of research. Reviewer 1 found 
the review on bias and representativeness helpful, so we decided to keep the length 
of the discussion on bias largely the same.

Comments 3: 
> Next, it is not clear how the merits of the five case studies are
> evaluated. Was it on the SETS framework or the use of multiple
> different VGI sources to reduce bias. Both are interesting, but the
> lack of clarity perhaps this suggests that the takehome message
> of this paper is not clear. They could both be in the paper but this
> needs to be better framed and presented.

Response 3: We agreed and specifically updated and shortened the Conclusions chapter
to better highlight our takehome message. This also reflects on similar comments from
Reviewer 3. In regard to the SETS framework, we updated Figure 1 to better
highlight that Geosocial Media/VGI indeed captures or contains biases from all
three poles. Therefore, the SETS framework functions as the basis to systematize
biases. We updated L205-L216 accordingly.

Comments 4:
> One last thing, I can appreciate the note that the full data
> acquisition is beyond the scope of this paper, but to my
> knowledge there is no public API for Instagram. When I looked at
> the python code, it seemed it was pulling in existing data rather
> then using the API. This is not a problem, but could the methods
> make clear what was existing data as compared to novel pull
> from an API. I suppose this also feeds back to a concern about
> the time window of the datasets. Since it is 2024, are the same
> APIs available to replicate the suggested workfellows or
> solutions in the manuscript?

Response 4:
We agree that data collection and API queries are an important topic, especially 
in the context of our article. We appreciate the opportunity to devote more attention 
to this topic in §3.2 (Data Collection and Preprocessing). In particular, 
we emphasize that it is very difficult or impossible to re-collect the same data 
at any point in time:

"Data collection for these studies was performed using the official application 
programming interfaces (APIs) provided by the platforms. APIs are challenging to 
work with. They often change on a weekly or monthly basis, are difficult to fully 
sample, and are often incompletely documented. For example, the Instagram, Reddit, 
and Twitter APIs have changed in ways that would make it very difficult or impossible 
to retrieve the data again in the same form at any given time. Transparency, reproducibility, 
and reusability are critical aspects in this area of research. Therefore, we follow 
a workflow outlined in [29] that allows us to share the data collected from the APIs 
without com-promising user privacy. Based on this workflow, the data has been transformed 
into a privacy-friendly format that allows quantitative analysis without the need 
to store raw data."

This does not invalidate our analysis. Rather, from today's perspective, we see a particular 
need to share data and results because they cannot be collected again. This also highlights 
the need for regulatory action to increase the transparency of large platforms, 
as is the aim of (e.g.) the EU's Digital Services Act.

We also highlight that the data that we collected and use to analyze results and figures are
fully available in a separate data publication:
Dunkel, A.; Burghardt, D. Supplementary materials for the publication "Assessing Perceived 
Landscape Change from Opportunistic Spatio-Temporal Occurrence Data". Opara digital research 
archive. DOI: 10.25532/OPARA-572.

Comments 5:
> Line 30 – since Land uses a numbered citation format could you
> not the year of this piece of work? (and at line 62)
> Line 33 – Perhaps in addition to human and phenomenal, the
> authors could add environmental? Or expand on why
> pheromonal is the best term here?
> The transition between line 42 and 43 could be expanded
> In line 135 the authors state “better reflect the user's own value
> system,” what is this better than?

Response 5:
Thank you for pointing out these additional errors and confusing passages. We have 
corrected all errors and rephrased sentences according to your suggestions. For convenience, 
we copy the lines referring to the last two comments:

LL41-45: "This includes temporal characteristics, trends, and collective perceptions of landscape 
change. Consequently, both the human viewer and the landscape are important issues 
in the assessment of scenic resources. In recent years, however, search and ranking 
al-gorithms and the global spread of information increasingly influence the behavior 
of large groups of people [4]. This affects collective engagement and interaction 
with the landscape and its scenic resources."

L135: "In summary, the above research suggests that opportunistic data tend to be 
better for inferring users’ subjective values, including individual preferences for 
activities and observational behavior, making them suitable for assessing landscape 
perception and scenic resources."

Comments 6:
> In line 150 the authors state “…categorizing three 150 broad
> umbrella biases found in opportunistic data: Ecological, Social,
> and Technological.” How do these source of bias differ or relate
> to the sources of bias noted above (line 106) and the two
> proposed solutions from the literature?

Response 6: Indeed this was not as clearly formulated. We updated lines 151-162.
Specifically, we use the Ecological, Social, and Technological domain to 
"systematize biases", which is largely missing in the existing literature cited.
In case studies, we show hwo to assign biases, such as noted above (line 106), to these
umbrella categories. For convenience, we copy the updated paragraph (LL151-162):

"This study presents five case studies. We discuss three main areas in which change 
can occur: Ecological, Social, and Technological. The domains are borrowed from the 
SETS framework. They are used to systematize biases in the case studies and to assess 
perceived landscape change from different perspectives. Rather than looking at a 
sin-gle dataset in detail, the cross section allows us to test the system under different 
parameters. In the literature cited above, effects of technology are often subsumed 
under distributional measures of the social domain or treated as one of many different 
biases affecting data collection. Explicitly considering technology as an independent 
component helps us systematize the analysis process and better distinguish important 
levers for biases found in the data. We show how the framework can help analysts 
disentan-gle the three domains when interpreting and making sense of temporal patterns 
in community-contributed opportunistic data sources."

Comments 7:
> The authors state at line 427 “The first two case studies showed
> relatively weak ecological coupling (E),” However one could
> argue in Norway and in the national parks there is a clear
> seasonal variation.

Response 7: Agreed, this was a poor introduction sentence to this case study and
was updated as follows:

"The first two case studies show a mix of ecological, technological, and social dynamics 
in data patterns. Is this always the case? To illustrate the impact of a single phenomenon 
across multiple platforms, we considered observations of cherry blossoms (E) shared 
on Twitter and Flickr."

Comments 8:
> I am not sure I an see in figure four “The rise and fall of Twitter”.
> Particularly given the fact that the data ends in 2017

Response 8: Thank you for spotting this mistake. Indeed, we used observations
for this statement that were not included in this article. The sentence was 
corrected as follows.

"The rise of Twitter, on the other hand, appears to be slightly offset, with a 
noticeable peak in 2014, according to our data. "

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Today, humanity is faced with an exponential increase in the amount of incoming information. Moreover, the information can be both reliable and false. This problem can no longer be ignored, including in environmental research and planning. An interesting and original research analysis of this problem is offered by the authors of the paper.  This study is fully in line with the theme of the journal Land and the special issue. I haven't got any big comments or concerns.

 The paper has a scientific novelty, which consists in an original approach to solving the problem.

 The research undoubtedly has theoretical and practical importance. The proposed techniques may help to draw attention to overlooked or underestimated patterns of landscape change, fill in missing data between periodic surveys, or corroborate and support field observations. Despite limitations, the results already provide a comprehensive basis for developing indicators with a high degree of timeliness for monitoring perceived landscape change over time.

 The relevance of the study is well justified in the introduction.  The current state of the problem is described acceptably.

 The methodology approaches are described in detail. The authors used methods adequate to the tasks set.

 The research results are presented consistently and well visualized. The paper contains 6 visual figures and 1 informative table.

The conclusions follow from the results and are reasonable. The paper will be of interest to a wide range of readers. The only thing I want to advise the authors to improve the paper is to formulate the main conclusions more clearly. Now this section is too big and the reader may not catch the main thoughts. This should be stated briefly and clearly.

Author Response

Comments 1: 
> Today, humanity is faced with an exponential increase in the
amount of incoming information. Moreover, the information can
be both reliable and false. This problem can no longer be
ignored, including in environmental research and planning. An
interesting and original research analysis of this problem is
offered by the authors of the paper. This study is fully in line with
the theme of the journal Land and the special issue. I haven't got
any big comments or concerns.
The paper has a scientific novelty, which consists in an original
approach to solving the problem.
The research undoubtedly has theoretical and practical
importance. The proposed techniques may help to draw attention
to overlooked or underestimated patterns of landscape change,
fill in missing data between periodic surveys, or corroborate and
support field observations. Despite limitations, the results already
provide a comprehensive basis for developing indicators with a
high degree of timeliness for monitoring perceived landscape
change over time.
The relevance of the study is well justified in the introduction.
The current state of the problem is described acceptably.
The methodology approaches are described in detail. The
authors used methods adequate to the tasks set.
The research results are presented consistently and well
visualized. The paper contains 6 visual figures and 1 informative
table.

Response 1: Thank you for this overall very positive review and the 
comments below. We especially appreciate the summary of important
contributions from the reviewer's perspective.

Comments 2: 
> The conclusions follow from the results and are reasonable. The
paper will be of interest to a wide range of readers. The only
thing I want to advise the authors to improve the paper is to
formulate the main conclusions more clearly. Now this section is
too big and the reader may not catch the main thoughts. This
should be stated briefly and clearly.

Response 2: 

We have carefully revised the conclusions to better highlight our contributions. 
In particular, we have significantly shortened this section by removing content and 
the last paragraph entirely. In addition, we have added a paragraph that briefly 
and clearly states our main findings (LL660-665) and refers to them in bold text 
in the following three paragraphs, where we provide more detailed conclusions on these
findings.

Author Response File: Author Response.pdf