Investigation of Development Applications: A GIS Based Spatiotemporal Analysis in the City of Sydney Area 2004–2022

Abstract

When proposing and reviewing new developments, urban planners, architects and the broader public must make well-informed planning decisions that fit within the broader urban context to foster a sustainable future and avoid costly and unnecessary redevelopment later on. There is often no comprehensive, publicly available and data-based spatiotemporal body of knowledge to help support these decisions. This paper uses the City of Sydney (CoS) as a case study to show how open data about individual development applications (DAs) can be used to build a critical spatiotemporal information framework to fill this gap and guide important city-shaping design and planning decisions. This research proposes a novel and broadly applicable methodology based on Python data analysis and mapping to extract and visualise spatiotemporal insights from DA data in terms of DA lodgement numbers and locations, DA estimated costs, DA proposed land use and application processing times. The results show a consistent decrease in DA lodgement numbers since 2008, likely accentuated by the COVID pandemic since 2020. This is contrasted by a steady increase in the median cost of DAs since 2005. Development hot spots are identified in the Sydney CBD and the suburb of Zetland, whereas the western and central parts of the local government area (LGA) were found to be lodgement cold spots consistent with higher concentration of heritage conservation areas. DAs proposing new uses fall primarily in the retail category, followed by commercial land uses between 2005–2011 and residential uses since 2012. Analysis of DA assessment time showed that 76% of DAs were approved or refused within 3 months, with a positive but limited correlation between estimated cost and assessment time. All charts and maps are made available in an online dashboard.

1. Introduction

Population growth and development are two major intertwining themes in Sydney’s growth [1]. Population more than tripled from 1911 to 2016 and is projected to continue growing in the future (Figure 1) [2,3,4]. To accommodate the growth, there have been many developments over the years in terms of infrastructure, as well as commercial, retail and residential uses, driven primarily by private interests.

Development applications (DAs) are familiar to Sydney residents. They often catch attention as a letter in the mailbox or a posted notification outside a property. Such public notification is a key part of the planning process to ensure neighbourhood awareness of nearby developments and to encourage community participation [5]. Most DAs are notified for a period of between 14 to 28 days within a radius of 25 to 75 m [5]. This time allows parties to make submissions regarding the DA to either support or object, and such submissions will then be considered in the assessment of the application. DAs since 2004 have also been published on the City of Sydney (CoS) website, making them accessible to any interested member of the public.

DAs are the main mechanism for implementing the planning process. Anyone can submit a development application and indeed, nearly 90% of developments are submitted by non-developers [6]. DAs encompass a wide range of proposed changes from minor home alterations to large mixed-use developments. As such, they represent the primary way in which the community contributes to making the city go from plan to reality. A deep interest in this area inspired the writing of this paper.

1.1. Problem Statement

While individual current DAs are publicly available, there is a gap when it comes to surfacing the broader patterns of development across the spatiotemporal gamut.

The planning framework has three fundamental components: framework plan, planning regulations and planning process [7]. The plan along with regulations give investors and community certainty in terms of what the city aims to become, while the process corresponds to the implementation of the vision through individual pieces of land.

In New South Wales (NSW), the framework plan and regulations have been extensively communicated by the state government to the public through news and government websites. The introduction of the NSW Planning Portal in 2016 and the launch of the NSW Spatial Digital Twin project make planning data even more accessible now in a variety of convenient formats [8,9].

When it comes to implementation however, information can be harder to come by. Taking the CoS for example, while development applications have been published online since 2004, each corresponds to an individual, out of context, snapshot in time. City monitor reports provide a once-a-year summary of recent developments, but the public is unable to access a rich, holistic view of development in the city over time. Information about ongoing developments is often limited to specific projects highlighted by the media or local associations.

At the scale of individual sites, built environment professionals engaged by landowners are the key to implementing the planning vision, but lack of information can be a handicap. Approval authorities typically expect developments to stick strictly to planning controls, unless they provide solid evidence for proposed design variations. In the absence of a comprehensive, easily accessible body of knowledge of the past, present and likely future changes of a place, architects and developers must choose between a lengthy, expensive and ultimately risky site analysis, or the simpler, more economical path of strictly sticking to planning controls. More often than not, the latter wins.

There is a clear gap: without a suitable, publicly available spatiotemporal body of knowledge, the planning process is effectively limited to planning controls. This paper aims to fill this gap, starting with an analysis of the broader insights that can be extracted from currently available data in the CoS local government area between 2004 and 2022 [10].

1.2. Research Aim and Objectives

The aim of this research is to use available CoS DA data to perform a spatiotemporal investigation of the developments in this LGA during the nominated period. This will contribute towards filling the gap of publicly available, comprehensive, data-based spatiotemporal information about development patterns in the CoS area.

The following questions will be explored:

• What is the spatiotemporal distribution of developments?
• Is there a spatiotemporal pattern of development spending?
• Is there a pattern of evolution in different categories of proposed development use?
• How do assessment times vary between DAs?

This study also aims to develop an interactive dashboard comprising charts and maps from this study.

The dashboard comprises:

• DA hot and cold spots
• DA assessment time patterns
• DA estimated cost
• DA proposed usage patterns

2. Literature Review

As Jane Jacobs noted decades ago, cities are organised complexity [11]. To analyse the more recent DAs, it is necessary to first understand the wider context. The literature review starts with exploring the importance of informed decision making, before moving on to describing the current development assessment authorities as well as the different development types in Sydney. It then discusses the recent COVID-19 outbreak, as it constitutes a major disruption event overlapping with the end of the study period. This section finishes with a review of GIS for mapping and understanding the development analysis.

2.1. The Need for Better Informed Development

As Robert Freestone noted, “planning scarcely features in general accounts of Australian history” and Sydney is no exception [12] (p. 73). The past half-century of planned development is only the beginning of a more organized city making approach. The value of more informed decision making when it comes to development is evident in the liveable city we experience today, as compared to its messy beginning. However, the benefit does not stop there.

According to the World Green Building Council, building and construction activities account for nearly 40 percent of world carbon emissions [13]. Common techniques to reduce the carbon footprint include sourcing local materials, reducing energy consumption or recycling materials. However, the more fundamental way to undertake systemic change is through the planning system, as current development controls often do not enable appropriate or sustainable uses, due to their delay in responding to the rapidly changing environment. As a result, sites may be developed in a way that cannot keep up with rapidly changing needs and need to be demolished and rebuilt within a relatively short time span. Preventing such short-lived projects from happening and instead providing longer-lasting buildings with in-built flexibility that can embrace future changes would greatly reduce resource consumption in the built environment.

A more easily accessible and comprehensive knowledge of the development history of the site and its context allows a better understanding of how the locality is likely to evolve into the future and enables innovative design ideas for developments that meet the current needs but stay relevant for the decades to come. This research aims to start providing such insights to the general public via a dashboard.

2.2. Current Development Assessment Authorities in Sydney

While the Environmental Planning and Assessment Act 1979 (EPAA) is still the backbone of the current NSW planning framework, a series of planning reforms have happened since, redirecting much planning power from the local government to the state [6,14,15]. As a result of this centralization of planning power, there are now three types of consent authorities in CoS: the City Council, Central Sydney Planning Committee (CSPC) and the NSW Planning Minister. The CSPC is composed of half Council and half State representatives, consisting of the Lord Mayor of Sydney, two CoS councillors, and four experts appointed by the NSW Planning Minister [16]. Figure 2 explains this power distribution with blue representing State power and orange representing Council power.

Most of the CoS LGA is under the approval of the Council or CSPC. This planning application pathway is usually via DA. While DAs with an estimated cost of more than AUD 50 million go to the CSPC, the majority of DAs are assessed by the Council. The other areas in the LGA fall under the major project planning pathway which will be explained in the next point. This paper will focus on Council-managed DAs, which represent most applications. It has the potential to expand to applications assessed by the State in the future.

2.3. Current Development Types in Sydney

As of 2022, there are several key development types in the CoS LGA.

2.3.1. Exempt Developments and Complying Developments

Firstly, not all development requires approval. Works such as ‘minor renovations and low-impact works’ fall under exempt development and do not need approval from authorities [17]. Complying developments involve larger works than the above, but only need sign-off from a certifying authority to obtain a complying development certificate (CDC) [18]. Examples of such development include building a granny flat, a swimming pool, or removing a tree. These two categories are not included in this study due to limited data availability and the minor nature of this type of work. Since July 2021, all CDC applications are to be lodged via the NSW Planning Portal [18]. As the records become substantial, they could be included in a future extension of this research.

2.3.2. Major Projects

On the other end of the spectrum are major projects. These projects are deemed to have State significance as a result of the size, commercial value or potential impact they may have [19,20]. This category of projects, now called State Significant Development (SSD) or State Significant Infrastructure (SSI), was introduced in June 2005 to promote economic development by enabling a dedicated and efficient state-led approval process [21,22].

These applications are not included in this study. This is because information for major projects is currently being transferred to a new website and is presently in two locations [20,23]. Neither website provides export to CSV functions. Web scraping is required to obtain a mass list of developments for analysis and they can be added later to enrich the study.

2.3.3. Development Applications

The next category is standard DAs, which make up the majority of applications [24]. They are the focus of this paper. CoS provides spatial information about developments via three platforms.

Firstly, single DAs can be viewed on the CoS website in a format similar to Figure 3 [25]. This platform provides a relatively isolated view of one DA at a time. Secondly, exhibited DAs are displayed as points on a map, similar to Figure 4 [26]. Clicking on the points provides more details about the DA and a link to the exhibition materials page. Once an exhibition ends, the point is removed from the map while new DAs are added. This platform therefore provides only a snapshot of a moment in the DA history.

Lastly, DA information is provided through CoS’s development monitor series, which provides a yearly overview of development via four monitoring reports: visitor accommodation, commercial and residential monitors and a housing audit [27]. The first three reports provided mapping of DAs until 2018 [27]. After a gap in the 2019 report, the same mapping for 2020 is now available via the CoS Data Hub https://cityofsydney.maps.arcgis.com/apps/webappviewer/index.html?id=ab67ecb8c37f4ae38ac77066e02b03fe (accessed on 24 July 2021), as an interactive map, refer Figure 5 [28]. This map provides a useful snapshot of development usage and status but is just a snapshot of one year in the DA history of CoS.

As a result of the above observations, there is no publicly available information that provides a spatial review of all DAs over time. This is the gap that will start to be filled by this research.

2.4. COVID-19 and Development

A major event that happened towards the end of the study period is COVID-19. On 11 March 2020, the World Health Organization (WHO) officially declared the coronavirus a global pandemic [29]. This major disruption has speeded up long term city transformations such as digitalization, a move to remote working and online services, and created a wide range of other side effects [30,31,32]. The changes in lifestyles and movement patterns have had different degrees of impact on different localities [33]. On a grander scale, projected population growth might be impacted, as net overseas immigration has been a main contributor to Sydney’s population growth for many decades [1,34,35,36]. All of the above shifts are likely to translate to changes in development patterns over time [1,35].

Due to the recency of the pandemic, the longer term impact of COVID on the built environment is yet to be reviewed. In a recent study, a comprehensive review of COVID articles on the Scopus platform revealed that urban design is the least covered research theme when it comes to the influence of COVID-19 on cities [37].

Although limited in data, this research will aim to start the exploration of this impact on the development side, to observe if changes in terms of development patterns are already appearing.

2.5. Development Investigation with GIS

Geographic Information Systems (GIS) is commonly seen as a suite of tools for ‘the input, storage and retrieval, manipulation and analysis, and output of spatial data’ [38,39]. Since the 1990s and the wide spread of the internet, GIS has evolved to become more user friendly and has attracted many users from the public domain [39]. Nowadays, the smart city and open data moves by many government agencies, and the availability of free GIS tools such as QGIS, have made GIS based analysis possible for anyone who is interested.

Furthermore, as the NSW ePlanning Portal gained momentum in recent years, lodgement of planning applications such as DAs is now centralised on this one platform [40]. Should this development information become available in GIS formats in the future, GIS based analysis of developments in Sydney would have even greater room for exploration.

This research is GIS based as it has a proven record in similar spatiotemporal research of development patterns [39,41,42,43,44]. It also allows potential future expansion of this study, should more development data be released by the government.

3. Methodology

This research has undertaken a GIS approach to analyse over 15 years of development application data in the CoS LGA with over sixty thousand development entries. Python is used for data preparation and analysis as it is a common programming language with a comprehensive collection of libraries, such as Pandas, Matplotlib and Plotly, for data analysis purpose [45]. Matplotlib and Plotly are similar in terms of functionality, but while Plotly produces better diagrams and interactive maps, Matplotlib is faster for more complex mapping. As a result, Plotly was used to generate all charts while Matplotlib was used to produce all maps. The proposed visualization methods for each research objective are detailed in

Table 1. Proposed methodology per research objective.

Research Question	Tool	Method
What is the spatiotemporal distribution of developments?	Python	Bar charts, linear charts, location mapping, hot and cold spot analysis, space time cube
Is there a spatiotemporal pattern of development spending?	Python	Bar charts, pie charts, linear charts, scatter plots, hot and cold spot analysis, quintile distribution mapping
Is there a pattern of evolution in different categories of proposed development use?	Python	Bar charts, pie charts
How do assessment times vary between DAs?	Python	Pie charts, scatter plots, bar charts

.

As a key thinking behind this study is to build the foundation for future, further analysis, JupyterLab is also used to give better structure and documentation to the Python code by keeping code for different processes in different notebooks (refer Figure 6 for the overall workflow). More details on data preparation and analysis will be covered later in this section.

Visualisation outputs were separated into Jupyter notebooks of their own for ease of management, as they will need to be used in this report and the compilation of the final dashboard. QGIS is also used to visualise some earlier context mapping for its ease of usage.

Finally, the charts and maps are embedded into a publicly accessible website using Plotly Dash.

3.1. Study Extent

The CoS LGA is comprised of 33 suburbs (including partial suburbs) (Figure 7) and is located in the east of NSW. Its estimated residential population as of 2021 is nearly 250,000 with a population density of over 90 persons per hectare, which is the highest in NSW [46,47].

There has been a considerable amount of development in the past years and the local council has made a large amount of data representing such development available online. There is also a reasonable amount of planning data available in GIS format online from the NSW state government and the local council. The availability of the above data makes this a suitable study for the timeframe of this paper.

3.2. Study Period

The online DA records cover the time range from the end of 2004 to the present. This means more complete DA data is available for 2005 to 2021. 2004 and 2022 data are incomplete at the moment but will be used where applicable and excluded when an equal comparison needs to be made between years. The current CoS LGA extent was established in early 2004 and has not changed since then [48].

3.3. Data Sources

The primary data source that forms the basis of this research is a CSV file downloaded from the CoS website. There are three main streams of data in this research: background data, development data and land zoning data.

3.3.1. Background Data

This set of data provides background for the analysis of other information (

Table 2. Background information.

Description	Data Type	Source	Processing Required
LGA boundary	Shapefile	CoS Data Hub	Import into QGIS and styled
NSW suburb/locality boundaries [49]	Shapefile	Data.gov.au	Processed
Planning consent authorities	Shapefile	CoS Data Hub	Import into QGIS and styled
Property boundaries [50]	Shapefile	CoS Data Hub	Import into QGIS to be joined with other data
Background map	WMS	Mapbox	Design in Mapbox and linked into QGIS

).

3.3.2. Development Data

CoS provides DA and footway usage application (FA) information on their website which can be downloaded as a single CSV file. DA information is available for all applications lodged after 8 November 2004 [51].

Details of the dataset can be found in

Table 3. Attributes of DA dataset.

Field Name	Description	Sample Value	Comments
App Number	DA number Development type/lodgement year/development number/subsequent development series number	D/2021/267 DU/2002/952/A	Can be used to identify related applications.
Address	Full address	84–110 Castlereagh Street\rSYDNEY NSW 2000	Needs cleaning up of the ‘\r’ string. Sometimes, geocoding can have an issue processing addresses with a hyphen between numbers
Street	Street address	Castlereagh Street	Not used as it overlaps with ‘Address’ field
Suburb	Suburb	ALEXANDRIA	Allows quick sorting of DAs into different suburbs for comparison
Description	Brief summary of DA	Demolition of the former Ryvita Factory and construction of a new, mixed use development comprising nine (9) interlinked buildings and the transfer of land at 19–25 Lyons Road, Camperdown to Council for community purposes.	Contains key information about the DA but needs to be extracted
Lodged Date	Date of lodgement	23 March 2021	Used to calculate the whole decision timeframe
Exhibition Closed Date	Date of exhibition closure	16 December 2004	Not used
Decision	Description of decision	‘Approved with Conditions’, ‘Refused’	Used to understand if the DA is approved or not
Officer	Names of DA officers	Names not to be shown	Not used
Est Cost	Estimated cost of DA	$3,919,948	Needs to be cleaned up and converted to numbers
Decision Date	Date of decision	13 September 2005	Used to calculate the whole decision timeframe
On Exhibition	If the DA is still on exhibition	Empty or ‘Yes’	Used to exclude DAs where no decision is made yet

.

3.3.3. Local Environmental Plan (LEP) Land Zoning Data

LEP zoning information is referenced in some of the analysis. Land zoning defines a primary use which is nominated for each piece of land. Each zone has permitted and prohibited uses, designed to promote government plans.

CoS experienced major planning changes in 2012, which was in the middle of the study period. Currently, the majority of the LGA is under Sydney LEP 2012, gazetted 14 December 2012. Figure 8 shows current planning controls that define land zoning. Land zoning information before this planning change is only available to the public in PDF format and will not be looked at in this study, so the study will focus on analysis that requires only the current zoning information.

A land use matrix prepared by the Department of Planning was also used to help with the categorization of DAs [52]. This is to prepare the DA usage information in a consistent manner to LEP categorization for ease of comparison.

3.4. Data Preparation

Data preparation is a key but time-consuming aspect in this study as the main data source contains lengthy text descriptions from manual inputs, which is highly susceptible to errors and inconsistencies in wording. The dataset was first reviewed manually to determine the preparation work required. The pre-processing was then implemented in Python so that it can be easily updated should more pre-processing be needed in future revisions of the dataset.

3.4.1. Data Cleaning

A data processing script was prepared in Jupyter notebook using Python to process and clean the dataset, in particular:

• fixing typos (Figure 9);
• fixing inconsistent punctuation and spacing (Figure 9 and Figure 10);
• fixing data types such as transforming monetary values into numerical values (Figure 11); and
• removing invalid records (Figure 12).

The initial cleaned dataset contains 60,795 entries.

3.4.2. New Fields

The DA number is first split to create two new columns, one for the base number and one for the subsequent version number. This is because all developments will have a unique DA number. Subsequent modifications to the DA will have a version letter added to the back of the original DA number. This allows grouping the original DA and all subsequent modifications together for analysis.

3.4.3. DA by Application Type

A manual review of the description field of the data reveals that not all developments are of equal extent. A good portion of the DAs are relatively minor, such as change of trading hours, or modification of development conditions. Therefore, the data is annotated step by step, applying keywords in the description field, to categorise the DAs for analysis. These keywords are referenced from LEP zoning uses and CoS city monitor report categories where applicable to make the datasets more comparable.

An additional column ‘Ignored’ is added in the same process, by assigning a ‘True’ value to the projects that are considered minor, and a ‘False’ value to the rest. This brings convenience later in the data analysis, when focusing on non-minor DAs where applicable.

3.4.4. DA by Proposed Usage

The dataset is scanned for keywords in terms of proposed usage. This process is automated based on repeated patterns in descriptions, such as “Change use from … to …”. A column called ‘New Use’ is added to capture the usage information extracted from the description field. 35% of DAs have a new use that can be detected through this method. A manual validation was done to check that the remaining 65% of DAs indeed do not have new uses proposed.

A Python script then maps each proposed use to a broader category. The categories are set based on available Council information, LEP zoning and work experience to capture common usage cases such as “Commercial”, “Retail” and so on. A list of all uses is then extracted from the ‘New Use’ column and manually assigned to each category. The list of categories is shown in Figure 13. In this way, 78% of the extracted proposed uses were able to be categorised.

3.4.5. Suburb and Property Boundaries

A search for an official shapefile of CoS suburbs was unsuccessful. In order to map CoS suburbs, the GeoPandas Python library was used to intersect the CoS LGA boundaries with NSW suburbs’ boundaries.

The property boundaries shapefile provided by Cos Data Hub has some inconsistencies that needed to be addressed. Eight addresses were found to have multiple property boundaries (Figure 14). They are deduplicated by merging them together. A few property boundaries were also found to overlap or be outside of the Sydney LGA. The intersection was used to exclude them. There were 31,838 property boundaries after this cleaning up.

In order to map each DA to geographical property boundaries, the ‘Address’ column is used to join each DA record to a property boundary using the GeoPandas Python library.

The majority of DAs were successfully matched. The remaining ones were found to have addresses that did not match the expected format (e.g., missing a postcode or “NSW”) (Figure 15). Further manual cleaning was not conducted as it only represents 0.6% of the dataset. These DAs are therefore excluded from geographical analysis at this stage.

3.4.6. Unique Counting of DAs

When one DA impacts multiple lots, each lot will have a row in the dataset. While this is good for looking at developments per lot, it is not suitable in cases such as counting the number of DAs or checking DA assessment times. In order to ensure that DAs encompassing multiple properties do not skew the analysis results, when it comes to the particular analysis, duplicates are removed beforehand.

When the address needs to be kept, such as when mapping hot and cold spots of estimated costs, the estimated cost column is instead updated to split it equally between multiple records. Consequently, the sum across records will be equal to the estimated DA cost expected.

3.5. Data Analysis

Data analysis is performed using a combination of bar charts, line charts, pie charts, scatter plots, maps, hot and cold spot mapping, and a space time cube. This section will explain the reasoning for the choices and how they have been used in the analysis.

3.5.1. General Charts and Maps

Standard mapping was first used to provide a general overview of the locations of DAs over the years, in exploring the first research question concerning the spatiotemporal distribution of development. They were produced in the Python matplotlib library.

For other analysis where the spatial quality is not essential, charts and graphs are used to present visually clear messages via simple representation. Other spatiotemporal analysis in terms of development were reviewed in the process as references to find suitable methods [39,41,42,44]. They are created in Python using the Plotly library.

The charts and plots are made interactive on the dashboard to allow easy exploration of data.

3.5.2. Spatial Autocorrelation and Hot Spot Analysis

When spatial quality is key to the understanding of patterns, such as in the case of cluster identification, spatial autocorrelation is examined. Spatial autocorrelation is ‘the correlation among values of a single variable strictly attributable to their relatively close locational positions’ [53]. As per Tobler’s first law of geography, ‘everything is related to everything else, but near things are more related than distant things’ [54]. Spatial autocorrelation has been demonstrated in case studies to be a suitable tool for clustering analysis [53].

Python is used with PySAL library to perform spatial autocorrelation at the global and local levels. The value of global Moran’s I ranges from −1 to 1, with −1 indicating dispersal and 1 indicating clustering [43]. It is useful in analysing if there is any sign of an overall spatial pattern present in the data and is therefore used as the first step in checking DA location and DA cost clustering in answering the first two research questions. A threshold distance-based weighted matrix was used with a 600-m distance, with the same dimension as the space time cube analysis. Similarly, local Moran’s I was analysed with the same weight matrix to identify local hot and cold spots across the geographical area studied. It helped to find areas that have been actively developed as well as areas that have remained largely undeveloped through the study period. All these results will be discussed in the next sections.

3.5.3. Space Time Cube

Swedish geographer Torsten Hägerstrand introduced time-based geography thinking in the mid-1960s, which seeks to consolidate space and time into a single model for analysis [55,56]. The space time cube was a key element in his methodology where the base of the cube represents the geography along the x- and y-axis, and the z-axis or the height of the cube represents time (Figure 16) [55,57].

The space time cube is used in this study to visualise DA data in time and space. It was done in Python using the Numpy and Plotly libraries. The cube bins are designed to be 600-m. This is because while a finer grain provides more information, an overly fine cube size would present visual difficulties in analysing overall patterns. The ABS data structure SA1 (Statistical Area 1) and SA2 (Statistical Area 2) were used as references to help determine this dimension [58]. It was found that the mean area of the 2021 SA1 blocks is 54,403 m², which equals a grid of around 233-m, while the mean area of SA2 blocks is 1,796,484 m², equalling a grid around 1340-m. Space time cube diagrams were then tested within this grid range, and it was determined that 600 m was a suitable size.

4. Results

4.1. DA Numbers and Locations

4.1.1. DA Numbers

First, an overview of DA numbers is produced to understand the scale of development over the years (Figure 17). Excluding the years where data is incomplete (2004 and 2022), the number of DAs lodged is overall gradually increasing from 2005 to a peak in 2008, growing from around 2500 to over 3000 DAs per year. The numbers fluctuated from 2008 until 2013, before showing a downward trend, plateauing at 2000 DAs per year for the last 3 years. For 2022, based on a month-by-month comparison to the same periods in previous years (Figure 18), it seems that the DA numbers will at least continue at this level, if not dropping further. Overall, it is worth noting that the proportion of DAs refused is very low throughout the years, falling well under 200 cases per year (Figure 17).

It is still difficult to tell if the low DA numbers since 2020 are directly linked to COVID-19. However, 2020 started with a similar number of DAs as previous years but dropped in comparison from April onwards (Figure 18). This timing is consistent with the beginning of the pandemic in Australia, with NSW cancelling major events from mid-March, and adding more restrictions until the first lockdown at the end of March [59]. As restrictions gradually eased over April, May and June, June and July saw an increase in DA numbers before declining again with the number of DAs lodged seeing a significant drop in August (Figure 18). The lodgement rate stayed low until December 2020 (Figure 18), when it recovered as NSW controlled the first wave of the COVID-19 pandemic.

One general trend across all years is that the months of January and February always experience low lodgement rates but quickly climb to a peak in March. The low point corresponds to the Christmas break period, while the pinnacle occurs when all projects resume and actively start on development proposals for the new year. Similarly, the December period consistently sees a relatively high lodgement rate as people rush to complete projects by end of year.

4.1.2. DA Locations

A quick look at the year-by-year mapping of DAs (Figure 19) reveals that there clearly are areas that always have DAs as well as areas with almost no DAs through 2005 to 2021. The former includes locations such as the CBD and Alexandria, and the latter includes places such as Glebe and Redfern. This will be further examined later in the hot and cold spot analysis.

The global Moran’s I for developed areas is 0.03, which shows that development areas do not follow a predictable pattern. Across the whole study period, it is obvious from the space time cube (Figure 20) and the hot spot analysis (Figure 21 and Figure 22) that the northern portion of the LGA has always been a development hotspot. This includes localities such as Sydney (CBD), Haymarket and Elizabeth Bay. Other hotspots are found in certain southern localities, but to a lesser extent. This includes mostly the suburbs of Alexandria and Zetland and the northern part of Rosebery. This is not surprising as the study period largely coincides with the 20-year Green Square urban renewal process, which is located at the junction of Zetland, Alexandria, Waterloo, Rosebery and Beaconsfield [60]. However, as the renewal process is coming to an end, most of the associated changes are either completed or under construction, hence the disappearance of hotspots in the more recent years.

On the other hand, a large proportion of cold spots are found in the west and centre of the LGA, such as the localities of Glebe, Newtown, Erskineville, Eveleigh and Redfern. A review of the heritage mapping in the LGA (Figure 23) shows that these areas coincide with concentration of heritage conservation areas, which tend to have more constraints on development [61].

Total development area in each LEP land use zone was also examined numerically. Half of the zones had DAs lodged for over 70% of the land. The top three zones were B5 Business Development zone (99.6%), B6 Enterprise Corridor zone (85.6%) and RE1 Public Recreation zone (84.9%). Both B5 and B6 are small zones so the high percentages are not surprising. RE1 has large areas across Sydney, and they have seen lots of development probably due to the government emphasis in providing more and better public spaces. It is worth noting that both R1, the General Residential zone and R2, the Low Density Residential are at the bottom of the list, with around 25% and 35% of the zone developed respectively. A review of such zones shows that they are largely occupied by small lot houses and are mostly cold spots in the DA location analysis above.

4.2. DA Estimated Value

4.2.1. Overview

Given that the numbers of DAs are similar across the years, attention is then paid to the estimated value of newly proposed projects to better examine the magnitude of DAs overtime. Figure 24 shows 2015 saw the biggest total investment in DAs, after which, while values fluctuate, a general declining trend is observed. The median development cost gradually increased from under AUD 20,000 in 2005 to around AUD 50,000 in 2012, then accelerated until 2015, and continues increasing, but at a much slower rate to 2021 (Figure 25). Most projects are around AUD 130,000 as of 2021. Figure 26 highlights the substantial gap between average and median, meaning there has always been some major investments during the study period. However, the number or scale of such projects has been decreasing over the last 5 years, as demonstrated by the drop in average estimated cost.

4.2.2. Investment Hot and Cold Spots

Hot and cold spots in terms of estimated project value indicate areas with larger or smaller amounts of capital investments for proposed developments. Figure 27 reveals a belt of such cold spots in the west and middle of the LGA, which is consistent with the mapping of DA lodgement cold spots earlier, but much larger in extent. The hotspots are in similar locations to the DA lodgement ones but less intense, mainly clustering around the CBD and Zetland areas. Detailed yearly review in Figure 28 further illustrates how the CBD is the only location attracting major projects throughout the study period.

The uneven distribution of development spendings between suburbs is more closely examined in Figure 29. The Sydney CBD area has undoubtedly seen the largest amount of DA investments across the study period. In some years, such as 2006, 2007, 2017 and 2018, its total estimated cost is over 50% of that of the whole LGA. Zetland is near the top from 2005 onwards, but gradually makes way for other suburbs such as Alexandria, falling to fourth place in 2020. Other suburbs such as Rosebery, Haymarket and Waterloo are mostly in the second tier, occasionally having a spike in values.

The above suburbs are in the lead in terms of total estimated cost mostly due to concentration of high value projects. When it comes to the more common development scenario, the mapping of median values in Figure 30 clearly shows that none of the suburbs are constantly in the lead.

4.3. Approved DA Usage

This section of the study will focus on DAs where a proposed use can be identified, as explained in Section 3.4.5. Only approved DAs will be examined as they form most of the dataset and have the highest chance of realization.

Over the years, the biggest portion of the above DAs fall within the retail category, followed closely by commercial and residential (Figure 31). The next two categories are utilities and community, after which other categories fall under 5% of total applications. These top five categories also tend to be on the top in the yearly reviews (Figure 32), with retail clearly in the lead, followed by commercial between 2005–2011, and then residential since 2012.

When reviewed with Figure 33, it shows that retail, residential, utilities, visitors accommodation and mixed use tend to have relatively smaller areas per project as compared to commercial, community, education, entertainment and health uses.

A closer look was taken into three use categories of interest: commercial, retail and residential. It can be observed that the number of approved DAs in each category remained relatively consistent over time (Figure 34) but both the total (Figure 35) and the average estimated cost (Figure 36) increased over the years. Spot check of values showed that the increase is generally above inflation rates, meaning projects in all three categories are getting more expensive. It is unsurprising that although retail had the highest numbers, the average costs are mostly below the other two categories, as retail proposals tends to be small scale developments such as new shops and restaurants.

Mapping the locations of the above three categories of DAs (Figure 37) shows the different spatial distribution of these uses. Retail and commercial uses are concentrated around similar areas such as the CBD, Alexandria and Zetland, while residential proposals are more dispersed throughout, except for areas such as the CBD, Camperdown, Eveleigh and Alexandria. Retail and commercial uses also show clustering in linear patterns, likely to be along major road corridors.

4.4. DA Assessment Time

Assessment time is another important aspect of DAs. It provides an indication of how reflective the DAs are to changing urban conditions.

Figure 38 shows that overall, a good proportion of DAs (48%) were determined between one and three months after lodging. Generally, around 76% of DAs reached a decision within a 3-month timeframe, and most DAs (94%) were determined within half a year of lodging. The above observed pattern is also relatively consistent on a yearly basis, indicating there is no direct correlation between total number of DAs to be processed and assessment time (Figure 39).

The average is always higher than the median (Figure 40), meaning the dataset distribution is positively skewed, whereas most assessment times in the dataset are lower than the average. This is consistent with the earlier findings. Both average and median assessment times oscillate within a small range from 2004 to 2014, before a sharp increase in 2015 (Figure 40). The average and median values seemed to decline since then but still fluctuate to the present. An apparent drop after 2021 is due to incomplete data in 2022.

Assessment time is then reviewed by type of determination (Figure 41) to assess possible correlations. As the court process is not managed by Council, court-related entries are removed from this analysis to better understand CoS assessment times. As ‘Deferred Commencement Activated’ is the follow up phase of ‘Deferred Commencement’, they are combined for the purpose of this analysis. Figure 42 dives into the three remaining categories and demonstrates that DAs approved with conditions tend to be the fastest, mostly around 50 days. This is followed by refusals at around 80 days. Deferred commencement projects take double the time or more with a less predictable timeframe. This is consistent with practical experience, where Council and applicant would usually engage in rounds of discussions and modifications to try to resolve issues to get to deferred commencement or before a formal refusal is issued.

A review was also done to check if there is any correlation between estimated project cost and assessment time. Figure 43 shows that there is a small increase in assessment time as estimated cost increases.

4.5. DA Dashboard

All the diagrams and maps from the analysis in this paper are embedded in a publicly accessible website using Plotly Dash. The website can be accessed through the following link: http://datashboard.com (accessed on 1 September 2022).

5. Discussion

This study aimed to perform a spatiotemporal investigation of the developments in the CoS LGA from 2004 to 2022 to contribute towards filling in the gap of publicly available, comprehensive, data-based spatiotemporal analysis of development patterns in the CoS area. The results have shown several macro-level patterns in recent CoS development.

5.1. Development as a Sign of Socioeconomic Activity

Ruming (2009) noted that developments are unevenly distributed in NSW and taking 2005 to 2006 for example, Inner- and Middle-ring Sydney had approximately 1000 DAs per council [21]. This is confirmed by the over 2000 entries found in CoS LGA for the same time period, which is double the average.

The research found that the number of DAs lodged has fluctuated over the years and followed a downwards trajectory since 2013, stabilizing in the last three years. Despite the shortage of data in 2022, a comparison with the same period in the past years so far shows that 2022 is likely to stay at the lower level in terms of DA lodgement. The analysis of total estimated cost of approved DAs shows a more complex pattern, with higher costs from 2013 to 2017 which may be the reflection of a recent concentration of development into larger, higher-cost projects. This is further reinforced by the steadily increasing median estimated cost of DAs approved each year.

The spatiotemporal analysis of development hotspots highlights a strong development push in specific suburbs (the Sydney CBD and Zetland), whereas other areas such as Glebe, Redfern or the northern part of Waterloo are cold spots of development. This result was consistent with a development intensity or DA count, as well as with a development value (estimated cost) analysis. Further investigation into the correlation between cold spots and social phenomena, in particular income levels and economic activity, could be a topic for future research.

5.2. DA Processing as a Constraint on Socioeconomic Progress

Approval time was argued by many as a key factor affecting housing affordability, as it reduces the potential pace of housing supply and therefore drives prices up [21]. The NSW planning and development assessment system has seen major reform since 2005 [21]. A key part of this transformation was to decrease development assessment times and costs [21]. Measures taken to achieve this included transferring planning power from local government to independent parties or state government, to remove local politics from the procedure [21].

Ruming (2009) found that average assessment time in Inner- and Middle-ring NSW LGAs was about 63 days, acknowledging that is probably due to the more complex investigation required on larger projects, rather than the result of intentional delay by government to hinder development activities [21].

The study of DA assessment time found consistent results and shows that the mean assessment time moved up further in 2015. The development process can still be a significant constraint, with more than 20% of developments requiring more than three months to be reviewed. On the bright side, almost 30% of developments are now reviewed under a month, and 4.24% under a week. While there is a positive correlation between estimated cost and assessment time, it was found that a significant number of high-value DAs were assessed with only short delays. Overall, 94% of developments are approved by the Council.

Further data would be required to understand the factors that influence the assessment time of specific DAs. One hypothesis to be explored in future research could be that sharing more development information as open data could result in better-prepared DAs and lessen the review load on the Council.

5.3. Evolving Uses as an Indicator of Changing Community Needs

New developments change the dynamics of the place, in particular when they introduce new uses. The research exhibited patterns of evolution in proposed use of new developments. Despite a relatively stable number of developments proposing new retail, residential and commercial uses, an upward trend of total estimated cost is seen from 2008 to 2019, with a stabilisation in 2020 presumably due to COVID.

As will be discussed below, the publicly available dataset offers insufficient information about current and evolving uses of developments, limiting insights in this area. The evolution of development uses may offer valuable insights for the community, as well as inform possible evolution of land zoning rules. This accentuates the need for the Council to release more detailed open datasets to the public.

5.4. Limitations

Throughout the research process, a range of limitations were recorded due to constraints such as data availability, time and cost. These limitations are discussed below.

5.4.1. Improving Data Availability

In recent years, government agencies nationwide started to participate in the open data movement by digitising data and making them publicly available online. However, a large amount of data are yet to be digitised, standardised and reviewed before being made available. As a result, historical planning data such as previous LEPs are not yet available in GIS format. Digitising such data is a labour-intensive process for private individuals and is therefore not done for this research. Once such GIS data is made available, they can be added to the study to complete the picture.

5.4.2. Property Boundary Data Availability

Another difficulty faced in this research is the accurate mapping of DAs. DA location information is currently only available as addresses in text format. The dataset needs to be processed to have DAs allocated on maps with the right site polygon boundaries.

The initial design was to geocode the DAs before mapping the geocoded dot file on lot polygons from NSW SIX Maps. However, the free geocoding options demonstrated considerable accuracy issues, and were either limited in number of entries allowed, or very slow in processing. Google Geocoding API seemed to be more accurate, but credits are required to be purchased. After investigation, this method was deemed unsuitable given the time and cost constraints of this study.

The other approach was to find cadastre files with address information so that data can be directly joined by the address attribute. The only publicly available option found was the CoS current property boundary. However, lot boundaries change over time due to land amalgamation and subdivision. To most accurately map DAs from 2004 to 2022, all historical lot boundaries in this period are needed. The only historical boundaries found on AURIN are PSMA cadastre for 2016 to 2020. FME Workbench was first used to compare geometries between adjacent years to observe changes, but it picked up negligible deviations, making it challenging to identify real lot boundary changes. Manual comparison of 2016 and 2020 cadastres by colour overlay shows that changes in these four years are minor in the overall scale of CoS. Furthermore, it was noticed that even the latest PSMA cadastre has many small deviations from the CoS lot boundaries. As DAs are also Council information, it was decided that it is best to use Council property boundaries for this study.

To confirm that this deviation from best practice would not have a large impact on the accuracy of the analysis, earlier DA datasets, such as 2004, 2005 and 2006, were checked against the CoS properties boundaries. There were less than 0.5% of the DA entries that could not be matched with a lot boundary and most, if not all of them were issues with the address. It was therefore concluded that the selected approach is suitable for the purpose of this study.

In the future, should Council digitise more of the historical property boundaries, the study can be adjusted to utilise the cadastre from the respective year to further refine the results.

5.4.3. Existing and Proposed Land Usage

The proposed land uses in this study were inferred from the DA description field using word pattern matching based on a manual review of the data, as described in Section 3.4.5. This process lacks precision and can lead to incorrect interpretations of results. More detailed proposed usage data can be found in PDF attachments on individual DA webpages, but it would require considerable data scraping to gather, which was outside of the capacity of this study. As exemplified in the city monitor map, CoS likely already has a structured dataset of proposed usage information [28]. Should Council add such information to this downloadable DA dataset in the future, the Python script can be updated to perform detailed proposed land use analysis.

When it comes to existing building usage, it has also proven difficult to find publicly available datasets. Google Maps provides the best information on this, but it is difficult to obtain as a workable dataset. Should existing land use information become easily accessible in GIS format, coupled with the above, the study can be extended to perform a broad range of land use investigations including land use evolution analysis.

6. Conclusions

This research designed a novel methodology for analysing individual DA data to extract spatial and temporal insights, making them available through a public dashboard. The research contributes towards filling the gap of publicly available, comprehensive, data-based spatiotemporal knowledge of development patterns.

With this case study of CoS DAs between 2004 and 2022, the approach focused on four research questions. The first two examined patterns of spatiotemporal distribution in terms of number of DAs or estimated costs. The third investigated patterns of evolution in development use such as residential, commercial and retail. The last question looked at assessment times and their possible influence on developments.

The study used in-depth Python analysis of CoS DA data combined with geographical property and suburb boundaries to produce charts and maps of macro-level development patterns. Moran’s I and spatial autocorrelation analysis helped highlight hot spots and cold spots at the level of individual suburbs. The analysis can be run again with updated datasets in the future to reveal further insights, making it a useful tool for other practitioners.

The results show a heavy development focus on specific suburbs such as the Sydney CBD and Zetland, both in terms of number of DAs and total estimated costs. Cold spots were identified in the west and the middle of the LGA, such as the localities of Glebe, Newtown, Erskineville, Eveleigh and Redfern, corresponding with a higher concentration of heritage conservation areas.

An investigation of new development uses highlighted rising costs across residential, commercial and retail developments, but insufficient information availability limited the depth of insights, especially when it comes to existing use which was missing from the dataset.

An analysis of DA assessment times across years and as a function of estimated cost showed a small but positive correlation between estimated costs and assessment time. It also indicated that although the majority of DAs are reviewed under three months, more than 20% require a longer time frame, which can be a significant constraint in the development process.

6.1. Further Applications

6.1.1. Broader Time Scale Investigation

As noted earlier in this paper, CoS DA information is currently available online from November 2004 onwards, limiting the scope of this study. Some of the older DA records (called Building Application or BA in earlier times) can be found at the CoS Archives & History Resources, where records assessed as being of permanent value are kept [62]. These records currently have limited information and there is no automated way to download the full list of such applications. Once these older DA records are better digitised, they can be used to expand the study scope to provide a fuller review of the Sydney development history.

6.1.2. Building Height Compliance and Other Detailed Investigations

DAs contain considerably more information than what is available in this open dataset. Missing information includes proposed use category, building height, number of levels above and below ground, number of car parking spots and number of apartments, etc. As exemplified in Section 5.4.3, most of this information can be found in PDFs on individual DA webpages but requires extensive data scraping to gather it. If CoS adds the above-mentioned information to this downloadable DA dataset, this study can broaden to provide spatiotemporal insights into areas such as building height compliance, residential density, housing typology distribution and car dependency.

6.1.3. Other Development Data

As mentioned early in this document, there are other types of developments in the CoS LGA, such as major projects. These large-scale projects, although not managed by the local government, often make a large impact on the LGA. Similarly, planning proposals that directly modify planning controls are another potentially interesting area of change to monitor in CoS. By combining these different pieces of the puzzle, a more complete picture of Sydney’s development pattern may be presented.

6.1.4. Wider Geographical Area

The process in this research is tailored to the data availability and format of the CoS LGA but has the potential to be adapted for other LGAs in NSW as well as other states. In fact, some local governments, such as Brisbane, directly provide proposed usage information [63]. This means the data processing could be fast-tracked in such cases. This implies that by adjusting the Python scripting, this study has the potential to expand to national level. Higher-level patterns across state and territories can help compare development approaches Australia-wide.

6.2. Recommendations

This paper revealed clear spatiotemporal insights based on a limited dataset of DAs in the CoS area between 2004 and 2022. While this outcome could only be achieved thanks to the availability of open datasets, in depth insights were limited by the quality and structure of the data, and in breadth by the limited spatiotemporal scope.

This highlights the importance of coordination at higher levels of government to design shared, well-structured open data formats and to encourage all LGAs to adopt them. Such efforts would enable powerful visualisations of spatiotemporal phenomena and help engage the community throughout the planning process, further strengthening a productive collaboration towards a common goal.