The Mass Profile of NGC 3268 from Its Stellar Kinematics

Abstract

The mass profile of the central galaxy of the Antlia cluster, NGC 3268, is studied through a spherical Jeans analysis, combined with a Bayesian approach. The prior distributions are derived from dark matter simulations. The observational dataset consists of Gemini/GMOS multi-object spectra observed from several programmes, supplemented with the kinematics of a small sample of globular clusters from the literature. An NFW mass profile and several options of constant anisotropy are considered. The analysis indicates a moderately massive halo, with a virial mass of (1.4 – 4.3)$\times {10}^{13}{M}_{\odot }$, depending on the assumed anisotropy. A comparison with the kinematics of the galaxy population from the Antlia cluster suggests that a fraction of galaxies is not yet virialised and may currently be infalling into the cluster.

1. Introduction

In the ruling paradigm, gravitational instabilities at early epochs led to the formation of the large-scale structure of the Universe, with massive clusters of galaxies inhabiting nodes, connected by filaments and sheets, where less massive systems reside [1]. Massive ellipticals (Es) are thought to have undergone two distinct formation phases [2,3]. The first corresponds to a burst of star formation at high redshift, lasting less than 1 Gyr [4], which resulted in a population of compact objects commonly referred to as red nuggets [5]. The second phase involves accretion and merger processes, which are responsible for building up the stellar and dark matter halos of massive Es, observed in the local Universe [6,7,8]. Although the influence of the environment on the physical processes experienced by galaxies is scarcely novel, e.g., [9], in recent years, the evidence has suggested that the influence of groups and clusters of galaxies extends beyond their virial radius [10,11].

Although the standard $\Lambda$CDM cosmological framework [12,13] remains the dominant paradigm in the community, owing to its success in explaining the cosmic microwave background, large-scale structure formation, and galaxy cluster dynamics, alternative theories of gravity are still actively explored. For instance, Modified Newtonian Dynamics (MONDs) proposes a modification of Newton’s second law at very low accelerations, which can naturally reproduce galaxy rotation curves [14,15]. Conversely, Modified Gravity (MOG or Scalar-Tensor-Vector Gravity) introduces additional gravitational fields to account for galactic and cluster dynamics without invoking dark matter [16]. These models underscore the ongoing debate regarding the true nature of dark matter and gravity. Nevertheless, $\Lambda$CDM remains the baseline scenario adopted in most astrophysical and cosmological studies, and the present work is developed within this framework.

Kinematical studies indicate that bright ellipticals in dense environments are embedded within massive dark matter haloes, e.g., [17,18,19], although the kinematical complexity in the cores of galaxy clusters may bias these results [20]. This view is further supported by the typical populations of their globular cluster systems (GCSs), which often comprise thousands of members, e.g., [21] (and the references therein), suggesting an extensive history of accretion and mergers. Hence, addressing the total mass of nearby structures is relevant for improving our understanding of the build-up of massive Es but also for disentangling the role of the environment in the evolution of galaxies in the surroundings of groups and clusters of galaxies.

The Antlia cluster, located at ≈35 Mpc [22], is among the nearest galaxy clusters, but it has been less extensively studied than others. It is described in the literature as dynamically young, with X-ray observations indicating a non-cool core [23,24]. Initial censuses of Antlia galaxy population were conducted by Hopp and Materne [25] and Ferguson and Sandage [26] and actualised in recent years by Smith Castelli et al. [27,28], with extensions to adjacent regions by Calderón et al. [29,30,31]. The total population is about 300 galaxies, including two confirmed compact ellipticals [32,33]. It comprises two subgroups dominated by the giant Es NGC 3258 and NGC 3268, both hosting rich GCSs [34,35,36], though some SBF distances might place NGC 3258 in the foreground [22].

2. Materials and Methods

2.1. Photometric Data

The photometric dataset consists of FORS1–VLT images in the V and I bands (programme 71.B-0122(A), PI: B. Dirsch). These images correspond to a single field containing NGC 3268, as well as NGC 3267 and FS90-175. The field of view is $6.8\times 6.8$ arcmin², with a pixel scale of 0.2 arcsec. The V and I images result from the combination of five single exposures of 300 s and 700 s each, respectively. In addition, a short 10 s exposure in I was used to avoid saturation in the central region of the galaxy. For further details on these images, see Bassino et al. [35].

In order to derive the surface brightness profile of NGC 3268 without contamination from its companions, an iterative fitting process was applied. First, the profile of the giant gE was obtained using the task ELLIPSE within iraf, after masking all bright objects in the field. Then, a model galaxy generated with the task BMODEL was subtracted from the original image. This second image, free of the extended light from NGC 3268, was used to fit the luminosity profiles of NGC 3267 and FS90-175, and their respective models were subtracted from the original image. The resulting image, containing only the extended light of NGC 3268, was used to derive its surface brightness profile, which was then subtracted from the original image to repeat the process. The final surface brightness profiles result from three iterations. These profiles were calibrated, applying the equations from Bassino et al. [35], and the extinction corrections were obtained from Schlafly and Finkbeiner [37]. The parameters corresponding to the profile in the I band, used to derive the stellar masses, are provided in Appendix A in Table A1.

Surface Brightness and Colour Profiles

The surface brightness profile of NGC 3268 in the I band, corrected for extinction, is shown in the top panel of Figure 1 as a function of the equivalent radius ($r_{\mathrm{eq}}$). The contamination level is estimated by fitting the outer profile at $r_{\mathrm{eq}}>175$ arcsec and is represented in Figure 1 by a horizontal dotted line. The filled symbols denote the surface brightness profile of the galaxy, corrected by contamination. The analysis reveals that the luminosity distribution is best reproduced by a three-component model, which has already been suggested by Huang et al. [38]. A model with fewer components results in systematic deviations in the residuals. The Sérsic law is chosen to fit the luminosity profile for each component, which follows the relation

$$I\left(r_{\mathrm{eq}}\right)=I_0·exp{\left({\displaystyle \frac{r_{\mathrm{eq}}}{r_0}}\right)}^{1/n}$$

(1)

where $I_0$ is the central surface brightness, $r_0$ is a scale parameter, and n is the Sérsic index characterising the shape of the profile, with $n=1$ corresponding to an exponential profile and $n=4$ describing a de Vaucouleurs law. In terms of surface brightness, and considering ${\mu }_0$ as the central surface brightness, it yields the expression

$$\mu \left(r_{\mathrm{eq}}\right)={\mu }_0+1.0857{\left({\displaystyle \frac{r_{\mathrm{eq}}}{r_0}}\right)}^{1/n}$$

(2)

The three fitted components are represented in Figure 1 by dashed curves, and their sum is represented by the dotted version. The residuals from the model are displayed in the bottom panel of Figure 1, and

Table 1. Best-fit parameters for the three-component Sérsic model of the I band galaxy profile.

	${\mathit{I}}_0$	${\mathit{r}}_0$	n
	${\mathrm{L}}_{\odot }{\mathrm{pc}}^{-2}$	$\mathrm{pc}$
Inner component	60,200 ± 13,000	$27.4\pm 9.2$	$0.42\pm 0.03$
Intermediate component	$180.0\pm 33$	$3700\pm 340$	$1.45\pm 0.09$
Outer component	$103.6\pm 9.5$	10,600 ± 1500	$1.37\pm 0.14$

lists the parameters of the three components.

Figure 2 displays the radial profiles of the parameters derived from ELLIPSE for NGC 3268 in the I band, together with its colour profile, as functions of $r_{\mathrm{eq}}$. The top panel shows the isophotal ellipticity ($ϵ$), and the bottom panel illustrates the position angle (PA, measured from north to east). Both parameters vary with radius, indicating that different galaxy components dominate the surface brightness profile at distinct $r_{\mathrm{eq}}$ ranges. Vertical grey lines mark $r_{\mathrm{eq}}\approx 15$ and ≈35 arcsec, corresponding to the radii where the surface brightness of the different components displayed in Figure 2 becomes significant.

2.2. Spectroscopic Data

The spectroscopic dataset consists of GEMINI–GMOS slit spectra, obtained as part of multi-object masks designed to study the galaxy and GC populations around NGC 3268. The raw data are publicly available in the Gemini Science Archive (programmes GS-2011A-Q-35 and GS-2013A-Q-37). The B600_G5303 grating, blazed at 5000 Å was employed, with small shifts in the central wavelengths applied to fill the CCD gaps. The slit width is 1 arcsec, yielding a full width at half maximum (FWHM) of $4.6$ Å. The wavelength coverage typically spans ≈ 4000–7200 Å, depending on the position of the slit. The total exposure time amounts to 3.5 h, with seeing conditions ranging 0.6–0.9 arcsec. Each scientific exposure is accompanied by calibration flats and CuAr arc spectra to correct for small variations due to telescope flexure. Flux standard stars were also observed on the same nights and with identical instrumental configurations to ensure accurate flux calibration. Data reduction was carried out using the GEMINI.GMOS package (Version 1.15) within iraf, following standard procedures. Further details of the reduction process are provided in Caso et al. [39].

2.2.1. Sky Subtraction

The mask design includes two slits covering the inner 25 arcsec of the galaxy. The remaining slits, located on the main body of NGC 3268, are centred on point sources previously analysed by Caso et al. [40]. Each slit typically encompasses 5–10 arcsec of the galaxy’s main body for subtraction, which serves as the scientific target in the present work. To minimise contamination from the central point source, only the slit sections located at distances greater than twice the seeing value were extracted for analysis. The spectral range adopted for the analysis is 4800–6000 Å, chosen to avoid the low signal-to-noise ratio at the blue end of the spectra and strong atmospheric features beyond the upper limit. This interval contains several prominent absorption lines that are particularly suitable for the aims of this study.

The resulting spectra include contributions from both the galaxy and the sky, which must be subtracted. For this purpose, the procedure described in Norris et al. [41] was followed, using a dataset with similar characteristics. First, the local background for the point sources was estimated from the number of counts in an annulus surrounding each target. This was performed on the VIMOS image in the V filter, whose spectral coverage is similar to the range indicated above. For each spectral pixel, the number of counts was normalised by the extent of the extracted region and compared with the number of counts measured for the local background from the photometry. As shown by Norris et al. [41], these quantities correlate, and a linear fit can be employed to extrapolate the number of counts to large galactocentric distances, where the galaxy contribution is negligible. For instance, the left panel of Figure 3 illustrates this correlation for 5300 Å. Applying this procedure to all spectral pixels yields a spectrum representing only the sky contribution. The ${\chi }^2$ values for the individual fits attain high confidence, $p>0.95$, except for wavelengths around 5576 Å, where a strong sky emission line, typically saturated in our GMOS observations, must be masked at a later stage. The right panel of Figure 3 displays the sky spectrum for the mask observed during 2011 in black, compared with the spectra extracted from three slits.

2.2.2. Definite Extraction and Kinematics Measurement

Although a signal-to-noise ratio (S/N) of approximately 5 is acceptable for measuring line-of-sight velocities ($V_{\mathrm{LOS}}$), estimating velocity dispersions (${\sigma }_{\mathrm{LOS}}$) requires $S/N\gtrsim 10$ [42]. To fulfil this criterion, the two slits covering the inner 25 arcsec of NGC 3268 are spatially rebinned to achieve $S/N\gtrsim 20$ at 5000 Å. Considering the typical seeing during the observations, the minimum spatial bin is set at 0.5 arcsec. The slits placed at larger galactocentric distances do not individually meet the $S/N$ criterion; hence, they are stacked according to similar galactocentric distances to achieve the required $S/N$. For this purpose, the $V_{\mathrm{LOS}}$ of individual slits was measured using the fxcor task within iraf, and small shifts were applied to correct for differences in $V_{\mathrm{LOS}}$ (see Section 3.1) that could otherwise lead to artificial line broadening and a spurious increase in ${\sigma }_{\mathrm{LOS}}$.

The mean $V_{\mathrm{LOS}}$ and ${\sigma }_{\mathrm{LOS}}$ for these rebinned spectra were measured using the penalised Pixel Fitting code [43,44,45] (ppxf). A subset of the E-MILES single stellar population models [46,47] was selected as templates for ppxf fitting, considering old populations (8 and 10 Gyr) with metallicities in the range $[\mathrm{M}/\mathrm{H}]=-0.4\mathrm{to}0.2$. The results are listed in

Table 2. Results from kinematical measurements by means of ppxf.

${\mathit{r}}_{\mathbf{gal}}$	$\Delta {\mathit{r}}_{\mathbf{gal}}$	${\mathit{V}}_{\mathbf{LOS}}$	${\mathit{\sigma }}_{\mathbf{LOS}}$
(arcsec)	(arcsec)	(km s−1)	(km s−1)
−4.1	0.7	$2654\pm 25$	$217\pm 5.0$
−3.2	0.7	$2658\pm 33$	$225\pm 5.4$
−2.5	0.5	$2670\pm 33$	$238\pm 5.2$
−2.0	0.5	$2675\pm 23$	$244\pm 5.3$
−1.5	0.5	$2672\pm 35$	$246\pm 5.5$
−0.8	0.5	$2702\pm 37$	$253\pm 5.7$
−0.3	0.5	$2750\pm 42$	$259\pm 6.1$
0.2	0.5	$2777\pm 50$	$260\pm 7.7$
0.9	0.5	$2818\pm 43$	$245\pm 7.9$
1.5	0.5	$2843\pm 44$	$242\pm 8.4$
2.0	0.5	$2844\pm 45$	$239\pm 8.2$
2.5	0.5	$2844\pm 40$	$238\pm 8.2$
3.0	0.5	$2829\pm 42$	$224\pm 8.3$
3.6	0.6	$2840\pm 46$	$222\pm 8.0$
4.4	0.9	$2836\pm 49$	$215\pm 8.2$
5.3	1.0	$2846\pm 44$	$212\pm 8.3$
6.3	1.0	$2841\pm 32$	$208\pm 8.3$
7.4	1.1	$2842\pm 32$	$205\pm 8.4$
8.7	1.2	$2823\pm 36$	$203\pm 8.9$
18.0	2.0	$2807\pm 31$	$198\pm 8.9$
20.0	2.0	$2798\pm 29$	$195\pm 9.1$
24.6	2.3	$2790\pm 28$	$192\pm 10$
39.0	6.0	$2816\pm 9.0$	$195\pm 12$
47.5	4.5	$2829\pm 9.7$	$192\pm 8.8$
80.0	5.0	$2832\pm 11$	$187\pm 20$
37.0	6.0	$2768\pm 8.9$	$199\pm 12$
41.0	5.0	$2697\pm 7.7$	$203\pm 13$
61.0	5.0	$2696\pm 9.0$	$201\pm 15$

.

2.3. Other Kinematical Sources

Caso et al. [40] present 23 spectroscopically confirmed GCs in NGC 3268. The sample extends to ≈47 kpc from the galaxy centre (≈4.6 arcmin at the distance of Antlia), with typical uncertainties of ≈20–40 km ${\mathrm{s}}^{-1}$. The weighted mean velocity for this sample is $2720\pm 40$ km s⁻¹, and the corresponding velocity dispersion is $220\pm 15$ km s⁻¹. The adjusted weighted Fisher–Pearson estimator for the kurtosis excess [48] yields $0.09\pm 0.14$, with the uncertainty estimated under the assumption of normality.

The projected density distribution of GC candidates is presented in Figure 4, based on the photometric catalogues of Caso et al. [36]. Only GC candidates brighter than $I_0=24$ mag were considered in order to ensure high completeness; see Figure 2 and Figure 3 in [36]. The black circles in Figure 4 were derived from deep FORS1–VLT photometry, whereas the grey squares were obtained from shallower MOSAIC–CTIO photometry, corrected by a multiplicative factor derived from the globular cluster luminosity function of NGC 3268. A Sérsic profile was fitted to the radial distribution and is shown as a red line in the figure. The best-fitting parameters are $n_{\mathrm{GCs}}=2.9$, $I_{0,\mathrm{GCs}}=16$ kpc⁻², and $r_{0,\mathrm{GCs}}=0.39$ kpc. The density counts at galactocentric distances below 5 kpc are not included in the profile fitting due to the increasing incompleteness that typically affects the photometry of extragalactic GCS in the central region of galaxies with high surface brightnesses when observed with ground-based telescopes, e.g., [49].

2.4. Dark Matter Haloes from a Numerical Simulation

The cosmological dark matter simulation MDPL2, part of the Multidark project [50] (publicly available through the official project database https://www.cosmosim.org/), was also used. This simulation spans a periodic cubic volume with a side length of $1{\mathrm{h}}^{-1}\mathrm{Gpc}$. It consists of ${3840}^3$ particles, each with a mass of $1.51\times {10}^9{\mathrm{h}}^{-1}{\mathrm{M}}_{\odot }$, and adopts the cosmological parameters of Planck Collaboration et al. [51]. The dataset corresponds to the catalogue of dark matter haloes identified with the rockstar halo finder [52] in the snapshot at $z=0$, representing the local Universe. Each halo is assumed to host a unique galaxy, with the main halo associated with the central galaxy of the system, while satellite haloes host the satellite galaxies.

In addition to the properties listed in the catalogue, an r band luminosity was assigned to each halo by means of a halo occupation distribution (HOD) method [53,54], following the prescription described in Caso [55]. Briefly, the galaxy luminosity functions fitted by Lan et al. [56], which distinguish between central and satellite galaxies, were adopted. An intrinsic scatter of $0.20$ in the stellar-to-halo mass relation was assumed when assigning $M_r$ magnitudes, in accordance with Girelli et al. [57]. The morphological type of the galaxy hosted by each halo was randomly selected as either early- or late-type, based on the environmental distributions observed by McNaught-Roberts et al. [58]. Further details on the procedure are provided in Caso [55].

2.5. Dynamic Modelling

2.5.1. Spherical Jeans Analysis

Under the assumption that NGC 3268 behaves as a non-rotating system (see Section 3.1) with spherical symmetry, the second velocity moments are related by the Jeans equation, e.g., [59,60], resulting in

$${\displaystyle \frac{d\left[j\left(r\right){{\sigma }_r}^2\left(r\right)\right]}{dr}}+{\displaystyle \frac{2\beta }{r}}j\left(r\right){\sigma }_r^2\left(r\right)=-j\left(r\right){\displaystyle \frac{GM\left(r\right)}{r^2}}$$

(3)

with $j\left(r\right)$ representing the three-dimensional density of the tracer population, $\beta$ is the anisotropy parameter, ${\sigma }_r\left(r\right)$ is the radial component of the velocity dispersion at the spatial galactocentric distance r, and $M\left(r\right)$ is the enclosed total mass. In the case of the fourth-order moments, if a distribution function of the form $f(E,L)=f_0\left(E\right)L^{-2\beta }$ with constant anisotropy parameter is assumed, then the Jeans equations are reduced to one of the forms (Łokas [59]; see Richardson and Fairbairn [61]):

$${\displaystyle \frac{d\left[j\left(r\right)\overline{{V_r}^4}\left(r\right)\right]}{dr}}+{\displaystyle \frac{2\beta }{r}}j\left(r\right)\overline{V_r^4}\left(r\right)=-3j\left(r\right){\sigma }_r^2{\displaystyle \frac{GM\left(r\right)}{r^2}}$$

(4)

with $\overline{{V_r}^4}\left(r\right)$ being the radial component of the fourth-order moment of the velocity.

The need for direct comparison with observations requires the projection of the velocity moments. The integration with the line-of-sight results, inverting the order of integration, yields the solutions

$${\sigma }_{\mathrm{LOS}}^2\left(R\right)={\displaystyle \frac{2G}{n\left(R\right)}}{\int }_R^{\infty }K(R,r)j\left(r\right)M\left(r\right){\displaystyle \frac{dr}{r}}$$

(5)

with $n\left(R\right)$ being the projected density of the tracer population, and R being the projected radius. The kernels $K(R,r)$ vary for different values of constant anisotropy, as indicated in the Appendix from Mamon and Łokas [62].

$$\overline{V_{\mathrm{LOS}}^4}\left(R\right)={\displaystyle \frac{2}{n\left(R\right)}}{\int }_R^{\infty }duGM\left(u\right)j\left(u\right)r^{2\beta -2}{\int }_R^{u}ds{\displaystyle \frac{GM\left(s\right)}{s}}{s}^{-2\beta }{K}^4\left(R,s\right)$$

(6)

with analogue kernels $K^4(R,s)$, presented in Appendix A from Caso [55] for constant anisotropy, ranging $-1/2\le \beta \le 1/2$.

$M\left(r\right)$ is defined as the sum of $M_{★}$ and the mass obtained from integrating the dark matter density profile. The analysis aims to determine, for a given model, the set of parameters that maximises the likelihood of reproducing the dataset by means of the cumulative probability of the ${\chi }^2$ distribution, with the corresponding percentile calculated from the expression

$${\chi }^2=\sum _j{\left({\displaystyle \frac{{\sigma }_{\mathrm{LOS},\mathrm{j}}^{\mathrm{obs}}-{\sigma }_{\mathrm{LOS},\mathrm{j}}^{\mathrm{pred}}}{e{\sigma }_{\mathrm{LOS},\mathrm{j}}^{\mathrm{obs}}}}\right)}^2$$

(7)

where ${\sigma }_{\mathrm{LOS},\mathrm{j}}^{\mathrm{obs}}$ represents the measurements of the velocity dispersion in the line-of-sight, as already presented in Section 2.2, and $e{\sigma }_{\mathrm{LOS},\mathrm{j}}^{\mathrm{obs}}$ corresponds to the associated error. In the case of ${\sigma }_{\mathrm{LOS},\mathrm{j}}^{pred}$, it symbolises the value predicted by the model for a specific set of parameters, and fixed anisotropy $\beta$.

For the GC sample, the set of parameters is determined by maximising the product of the individual probabilities for each object. These probabilities are obtained from the convolution of the observational $V_{\mathrm{LOS}}$ distribution and the predicted distribution at $R_{\mathrm{gal}}$. The observational distribution is modelled as a Gaussian centred on the measured $V_{\mathrm{LOS}}$ with a dispersion equal to the measurement error. The predicted distribution is centred on the mean $V_{\mathrm{LOS}}$ of the galaxy, 2750 km s⁻¹, which does not significantly differ from the mean $V_{\mathrm{LOS}}$ of the GCs sample. It is expressed in terms of Gauss–Hermite polynomials [63], with the terms describing asymmetric deviations from the normal distribution vanishing under the assumption of spherical symmetry.

$$f\left(w\right)={\displaystyle \frac{1}{\sqrt{2\pi }\sigma }}\exp \left(-{\displaystyle \frac{1}{2}}{w}^2\right)\left[1+{\displaystyle \frac{h_4}{\sqrt{24}}}(4w^4-12w^2+3)\right]$$

(8)

plus higher-order terms, which are assumed to be negligible in this approach, $w=(V_{\mathrm{LOS}}-V_0)/{\sigma }_{\mathrm{LOS}}$, and $h_4$ is related to the fourth-order moment of the velocity in the line-of-sight by $h_4=(\overline{V_{\mathrm{LOS}}^4}/{\sigma }_{\mathrm{LOS}}^4-3)/\left(8\sqrt{6}\right)$, with ${\sigma }_{\mathrm{LOS}}$ and $\overline{V_{\mathrm{LOS}}^4}$ arising from the Jeans analysis for each pair of parameters, $r^{\prime }$ and $c^{\prime }$.

2.5.2. Bayesian Treatment and Prior Distribution

This work employs a Bayesian approach to estimate the mass profile of the galaxy. The components of the observational dataset $D_{\mathrm{kin}}$ have already been described, both for the GC sample and for the galaxy kinematical data. The Bayesian analysis, therefore, aims to determine the probability $p\left(r^{\prime },c^{\prime }|D_{\mathrm{kin}},I_{\mathrm{prior}}\right)$, where $r^{\prime }$ and $c^{\prime }$ are the parameters of the model adopted to describe the dark matter halo, which, in the case of spherical symmetry, usually correspond to a scale radius and either a density or a concentration parameter. The information not contained in the dataset, which shapes the prior distribution, is represented by $I_{\mathrm{prior}}$. According to Bayes’ theorem,

$$p\left(r^{\prime },c^{\prime }|D_{\mathrm{kin}},I_{\mathrm{prior}}\right)={\displaystyle \frac{p\left(r^{\prime },c^{\prime }|I_{\mathrm{prior}}\right)·p\left(D_{\mathrm{kin}}|r^{\prime },c^{\prime },I_{\mathrm{prior}}\right)}{p\left(D_{\mathrm{kin}}|I_{\mathrm{prior}}\right)}}$$

(9)

with $p\left(D_{\mathrm{kin}}|I_{\mathrm{prior}}\right)$ assumed to be a normalisation factor, which is not calculated. The factor $p\left(r^{\prime },c^{\prime }|I_{prior}\right)$ represents the prior distribution for a specific set of parameters. The second factor in the equation includes both the probability obtained from the ${\sigma }_{\mathrm{LOS}}$ profile of the stellar population of NGC 3268 and that from the $V_{\mathrm{LOS}}$ distribution of GCs, and this can be factorised as

$$p\left(D_{\mathrm{kin}}|r^{\prime },c^{\prime },I_{\mathrm{prior}}\right)=p\left(Gal|r^{\prime },c^{\prime },I_{\mathrm{prior}}\right)·p\left(GCs|r^{\prime },c^{\prime },I_{\mathrm{prior}}\right)$$

(10)

The information contained in $I_{\mathrm{prior}}$ was assembled from central haloes in the MDPL2 simulation, whose galaxies have absolute magnitudes in the r band that are consistent with those of NGC 3268 and are classified as early-type galaxies (see Section 2.4). The total apparent R band magnitude of NGC 3268 is taken from Calderón et al. [31] and converted to the r band using the expected magnitudes in both filters for SSPs with solar metallicity and an age of 10 Gyr, derived from the MILES database. From this magnitude and the distance presented in Section 1 (corresponding to a distance modulus of $m-M=32.74\pm 0.14$ mag), we obtain the distribution of absolute magnitudes ($M_r$), which is used to statistically select haloes from the MDPL2 simulation on the basis of their $M_r$ assigned through the HOD method. Figure 5 shows the distribution of virial masses ($M_{\mathrm{vir}}$) obtained for the NGC 3268 analogues in the MDPL2 simulation, with the 95th percentile at $M_{\mathrm{vir}}=6.1\times {10}^{13}{\mathrm{M}}_{\odot }$. This probability corresponds to $p\left(M_{\mathrm{vir}}\right|I_{\mathrm{prior}})$, which is related to $p(r^{\prime },c^{\prime }|I_{\mathrm{prior}})$ through the expression

$$p(r^{\prime },c^{\prime }|I_{\mathrm{prior}})=p(r^{\prime },c^{\prime }|M_{\mathrm{vir}})·p\left(M_{\mathrm{vir}}\right|I_{\mathrm{prior}})$$

(11)

3. Results

3.1. Kinematics of the Stellar Population in NGC 3268

Figure 6 shows the $V_{\mathrm{LOS}}$ as a function of $r_{\mathrm{gal}}$, expressed in kiloparsecs, for the photometric major axis of NGC 3268, based on the two innermost slits (filled symbols) and a few other slits aligned with this direction (open symbols). Considering the distance assumed in Section 1, the scale is 170 pc arcsec⁻¹. The rotation curve is fitted through a function of the form

$$V_{\mathrm{LOS}}\left(r_{\mathrm{gal}}\right)=V_{\mathrm{sys}}+V_0·\left({\displaystyle \frac{r_{\mathrm{gal}}}{r_{\mathrm{gal}}+r_c}}\right)·exp\left({\displaystyle \frac{r_{\mathrm{gal}}}{r_t}}\right)$$

(12)

which is represented in the figure by a dashed curve. This function includes a factor that modulates the initial rise in $V_{\mathrm{LOS}}$, together with an exponential decline at large galactocentric radii, in order to reproduce the fading of the rotational component. The parameter $V_{\mathrm{sys}}=2750\pm 20$ km s⁻¹ represents the systemic velocity of NGC 3268, while the rotation curve reaches a maximum amplitude of $\Delta V_{\mathrm{LOS}}\approx 90$ km s⁻¹ at a galactocentric radius of ≈740 pc.

The ${\sigma }_{\mathrm{LOS}}$ profile of the stellar population in NGC 3268 is shown in Figure 7 as a function of galactocentric distance. Filled symbols denote ${\sigma }_{\mathrm{LOS}}$ values for bins corresponding to the two innermost slits, whereas open symbols represent measurements obtained by stacking slits with similar galactocentric radii. Emsellem et al. [64] introduced ${\lambda }_{\mathrm{R}}$ as a kinematic classifier that quantifies the importance of the rotational support for two-dimensional stellar kinematics, acting as a luminosity-weighted proxy for the projected angular momentum. For long-slit data, the parameter can be adapted following the prescription of Coccato et al. [65] and Salinas et al. [66]. From the measurements listed in Table 2 and the surface brightness profile described in Section 2.1, we obtain ${\lambda }_{\mathrm{R}}=0.16$ within $0.25r_{\mathrm{eff}}$. A fair comparison requires estimating ${\lambda }_{\mathrm{R}}$ out to $r_{\mathrm{eff}}$. Assuming that ${\sigma }_{\mathrm{LOS}}$ remains at a minimum of 190 km s⁻¹ up to $r_{\mathrm{eff}}$, and that $V_{\mathrm{LOS}}$ follows Equation (12), we obtain ${\lambda }_{\mathrm{R}}=0.12$. This result indicates mild rotation in NGC 3268, although we assume that spherical Jeans modelling can still be applied in this case.

3.2. Stellar Population Synthesis

If no spatial binning is applied, the two innermost slits of NGC 3268 provide an $S/N$ that is high enough to derive stellar population parameters through full spectral fitting with ppxf. The $S/N$ level also enables us to extend the spectral range towards shorter wavelengths, down to 4500 Å, which is particularly relevant for accurately constraining stellar ages. As in the kinematic analysis, the E-MILES stellar population models [47] are employed as templates, adopting a Salpeter initial mass function (IMF), which has been shown to be appropriate for representing the IMF in massive elliptical galaxies, e.g., [67,68].

For the inner 10 arcsec, the weighted mean age and metallicity are 10 Gyr and $[\mathrm{M}/\mathrm{H}]=0.01$, respectively, with the latter decreasing to $[\mathrm{M}/\mathrm{H}]=-0.18$ at galactocentric distances of 17–27 arcsec. The mass-to-light ratios derived with ppxf in the r band ($M/L_r$) are 5.8 and 4.1, respectively. The weights assigned by the algorithm to the set of single stellar populations are displayed in Figure 8 for both slits. These results are consistent with the colour profile presented by Huang et al. [38], which indicates colours typical of old populations with sub-solar metallicities, except in the innermost 10 arcsec, where the colours appear redder.

3.3. Kinematical Fitting of the Data

The dark matter density profile is represented by an NFW profile [69], which is commonly adopted in the literature to model dark matter haloes. We are aware that results of the kinematical fitting may depend on the adopted model, but large-scale studies of massive ETGs have proven that models producing cored haloes do not improve the goodness of fit, e.g., [18,70]. Furthermore, analysis of the dark matter density slope in the innermost regions of massive ETGs are consistent with the expectations from an NFW profile [71,72]. It is characterised by two parameters: a scale radius ($r_{\mathrm{s}}$) and a concentration parameter ($c_{\mathrm{vir}}$), defined as the ratio between the virial ($r_{\mathrm{vir}}$) and the scale radius [73]. In this formulation, the mean density within $r_{\mathrm{vir}}$ is equal to ${\Delta }_{\mathrm{vir}}\left({\mathrm{\Omega }}_{\mathrm{m}}\right)$ times the mean matter density (${\rho }_{\mathrm{m}}$). For ${\mathrm{\Omega }}_{\mathrm{m}}=0.307$, and when using the approximation given by Bryan and Norman [74], it yields ${\Delta }_{\mathrm{vir}}\approx 333$.

Following Equation (9), the joint probability for each pair of parameters $(r_{\mathrm{s}},c_{\mathrm{vir}})$ is obtained as the product of Equations (10) and (11). Although more sophisticated treatments of orbital anisotropy have been proposed in the literature [62], in this work, we restrict the analysis to solutions with a constant anisotropy parameter $\beta$. For massive ellipticals in the SAURON project [75], $\beta$ values are within one effective radius range from ≈−0.2 to ≈+0.2 (see their Table 2).

From this, Figure 9 displays the probability distributions for the parameters $(r_{\mathrm{s}},c_{\mathrm{vir}})$ in the isotropic case (i.e., $\beta =0$), adopting the prior distribution described in Section 2.5.2. The parameter grid is sampled with steps of $\Delta c_{\mathrm{vir}}=1$ and $\Delta r_{\mathrm{s}}=1$ kpc, covering a broad range of $M_{\mathrm{vir}}$. The colour gradient, from light yellow to dark blue, traces increasing joint probability, combining the results from both the stellar population and GC datasets. The side histograms show the marginal distributions of $c_{\mathrm{vir}}$ and $r_{\mathrm{s}}$. The maximum likelihood solution is found at $r_{\mathrm{s}}=67\pm 14$ kpc and $c_{\mathrm{vir}}=11\pm 1.7$, corresponding to a halo mass of $M_{\mathrm{vir}}=2.2\pm 0.4\times {10}^{13}{M}_{\odot }$. The mean values from the marginal distributions indicate a slightly more massive halo, with $r_{\mathrm{s}}=69.4\pm 11$ kpc and $c_{\mathrm{vir}}=11.7\pm 1.1$, and dispersions of ${\sigma }_{r_{\mathrm{s}}}=27.7\pm 1.9$ kpc and ${\sigma }_{c_{\mathrm{vir}}}=2.7\pm 0.4$. For the $M_{\mathrm{vir}}$ distribution, the mean and standard deviation are $1.6\pm 0.5\times {10}^{13}{M}_{\odot }$ and $1.0\pm 0.2\times {10}^{13}{M}_{\odot }$, respectively. Uncertainties were estimated from the parameter dispersions obtained from 100 Monte Carlo realisations of artificial samples, generated to mimic the observational dataset and its uncertainties, and these were analysed using the same procedure applied to the observational data.

A set of models with mild constant anisotropies, ranging from $\beta =-0.2$ to $\beta =0.2$, was also explored. The resulting trends are consistent, indicating increasingly massive haloes for higher values of $\beta$. The most probable solutions correspond to virial masses in the range $M_{\mathrm{vir}}=(1.4\pm 0.3-4.3\pm 0.6)\times {10}^{13}{\mathrm{M}}_{\odot }$, associated with larger-scale radii ($r_{\mathrm{s}}$) and lower concentrations ($c_{\mathrm{vir}}$).

Even though the galaxy population of the Antlia cluster was not used as a tracer, it is instructive to compare the expected ${\sigma }_{\mathrm{LOS}}$ profile from the most likely NFW parameters with the $V_{\mathrm{LOS}}$ distribution of the galaxy sample listed in Caso and Richtler [76]. First, a Sérsic profile is fitted to the projected radial distribution of galaxies, using the photometric catalogue from Calderón et al. [31]. Probable background objects, late-type galaxies, and galaxies within 10 arcmin of NGC 3258—the other dominant gE in the cluster—are excluded. The resulting parameters are $n_{\mathrm{gal}}=2.6$, $I_{0,\mathrm{gal}}=0.4$ arcmin⁻² and $r_{0,\mathrm{gal}}=2.9$ arcmin. Next, the expected ${\sigma }_{\mathrm{LOS}}$ at the galactocentric distance corresponding to the median of the spectroscopic sample (≈170 kpc) was calculated. For this, the NFW parameters that maximise the probability were adopted, yielding 270 and 345 km s⁻¹ for the isotropic case and $\beta =0.2$, respectively. Figure 10 shows the $V_{\mathrm{LOS}}$ for early-type galaxies as a function of projected distance from NGC 3268 ($r_{\mathrm{gal}}$). Spirals and galaxies projected within 10 arcmin of NGC 3258 are excluded. Circles, triangles, and squares correspond to dwarf, lenticular, and elliptical galaxies, respectively. The dark grey region is bounded by the caustic curves calculated using the infall model of Praton and Schneider [77], assuming ${\mathrm{\Omega }}_0=0.3$, which is the expected ${\sigma }_{\mathrm{LOS}}$ for the isotropic case, and $r_{\mathrm{vir}}=740$ kpc derived from the NFW parameters. The wider light grey region represents the analogous case for mild radial anisotropy, with $r_{\mathrm{vir}}=930$ kpc. In both cases, the caustic curves reasonably trace the distribution of early-type galaxies in Antlia, although there appears to be an excess of galaxies with large peculiar velocities compared to the $V_{\mathrm{LOS}}$ of NGC 3268. This is also evident in the side panel, which presents the histogram of $V_{\mathrm{LOS}}$ for the galaxy sample.

4. Discussion

The total mass of the Antlia cluster can be estimated using several methods. The galaxy NGC 3268 hosts a large population of GCs, of the order of thousands [35,36], reflecting a rich history of merger events, which is in line with the two-stage scenario for the build-up of GCSs in galaxy clusters, e.g., [3,78]. In this context, several studies have analysed the mean ratio between the mass in GCs and the halo mass, ${\eta }_{\mathrm{M}}=M_{\mathrm{GCS}}/M_{\mathrm{vir}}$, finding ${\eta }_{\mathrm{M}}\approx$ 3–4$\times {10}^{-5}$[79,80,81]. Assuming an average $M_{★}$ for GCs associated with galaxies of similar luminosity to NGC 3268 of $\approx 2\times {10}^5{M}_{\odot }$ [82], the halo mass estimated from the total GC population ranges (2.4–5.5)$\times {10}^{13}{M}_{\odot }$. This is slightly higher than the $M_{\mathrm{vir}}$ obtained in Section 3.3 for the isotropic case, but the difference is within the uncertainties. For cases with mild radial anisotropy, the results are consistent with the mass range inferred from the GCS population.

By means of X-ray observations with ASCA for NGC 3268, Nakazawa et al. [83] estimated a mass of $2.1\times {10}^{13}{M}_{\odot }$ within $\approx 275$ kpc, under the assumption of isothermality. For the parameters that maximise the probability in Section 3.3, the mass enclosed within the outer galactocentric distance considered by Nakazawa et al. [83] is $1.1\times {10}^{13}{M}_{\odot }$ in the isotropic case and ranges from $(0.8$–$1.8)\times {10}^{13}{M}_{\odot }$ for mild anisotropies, significantly lower than the estimate of Nakazawa et al. [83]. More recently, Wong et al. [23] derived $r_{200}\approx 900$ kpc from the X-ray temperature of the cluster and the scaling relation of Sun et al. [84]. Using the definition of $r_{\mathrm{vir}}$ adopted in Section 3.3, this $r_{200}$ translates into $r_{\mathrm{vir}}\approx 760$ kpc, which is in very good agreement with the value obtained for the most probable set of parameters, $r_{\mathrm{vir}}\approx 740$ kpc.

Applying a mass estimator to the population of early-type galaxies within 18 arcmin of NGC 3268, Caso and Richtler [76] derived a total mass of the subgroup centred on this galaxy of $(5$–$8)\times {10}^{13}{M}_{\odot }$. This value is more than twice the virial mass obtained in this work for the isotropic case, although it partially overlaps with the estimates for mild radial anisotropy. It should be noted, however, that the large substructure identified by Caso and Richtler [76] in the core of the Antlia cluster might bias the mass towards higher values. In this context, Calderón et al. [31] fitted a linear relation to the colour–magnitude diagram of the early-type population and found that galaxies with large peculiar velocities relative to the $V_{\mathrm{LOS}}$ of NGC 3268 are typically bluer. This suggests that such galaxies may have evolved in low-density environments, e.g., [85,86], and are therefore unlikely to be fully virialised within the cluster potential.

Hence, the comparison with estimates and scaling relations from the literature agrees with the results of this paper, favouring some degree of radial anisotropy to yield a more massive halo.

5. Conclusions

This paper aimed to derive the parameters of the NFW profile that provided the best fit for the kinematical dataset, confirmed by velocities and dispersions in the line-of-sight of the diffuse stellar population, as well as velocities for a small sample of confirmed GCs. The use of Bayesian statistics based on prior distributions derived from dark matter simulations, plus the Gauss–Hermite series of velocity distribution in the line-of-sight, emerges as an alternative method to constrain the parameters of the mass profile when the number of halo tracers is limited.

Several constant-anisotropy models were assumed, and the most likely NFW halo was selected in each case, yielding a moderately massive halo with a virial mass ranging from $1.4\pm 0.3$ to $4.3\pm 0.6\times {10}^{13}{M}_{\odot }$. Further comparison with observational evidence from various sources in the literature suggests that NGC 3268 may lie at the upper end of this range, which implies some degree of radial anisotropy for the galaxy.