Threads: Engagement and Replies

Overview

Buffer is launching a new feature called Community that will help creators quickly respond to comments on several different platforms. In this analysis we’ll look at the effect that replying to comments on Threads has on engagement.

What We Found

When we compare each Threads account to itself over time, posts with comments that have been replied to outperform the account’s own baseline by roughly 42%, on average. This result holds even when we restrict the sample to posts that received at least one comment.

An analysis of Z-scores also suggests that replying to comments has a positive impact on engagement. Posts with replied comments tend to sit above an account’s typical engagement level, while posts without replied comments sit slightly below.

The main takeaway is that replying in the comments is associated with a meaningful lift in engagement. This isn’t an unexpected result, but it’s nice to see it borne out in the data.

Data Collection

The SQL below returns around 129 thousand Threads posts sent in the past two years as well as the number of comments (replied and unreplied) that they received.

sql <- "
    select
        u.id as post_id
        , u.profile_id
        , u.user_id
        , u.sent_at
        , u.likes
        , u.replies
        , u.reposts
        , count(distinct c._id) as total_comments
        , count(distinct case when c.status = 'replied' then c._id end) as replied_comments
        , count(distinct case when c.status = 'unreplied' then c._id end) as unreplied_comments
    from dbt_buffer.publish_updates as u
    inner join dbt_buffer.community_comments as c
      on u.id = c.post_id
    where u.profile_service = 'threads'
      and u.sent_at is not null
    group by 1,2,3,4,5,6,7
"

# get data from BigQuery
posts <- bq_query(sql = sql)

Data Preparation

First we’ll construct a few core metrics and indicators, then filter out posts with no engagement data. We’ll then calculate a couple summary statistics to get an idea of what our data looks like.

# construct core metrics and indicators
posts <- posts %>% 
  mutate(
    engagements = likes + replies + reposts,
    has_replied_comments = replied_comments > 0,
    has_any_comments = total_comments > 0,
    has_any_replies = replies > 0,
    log_engagements = log1p(engagements)
  ) %>% 
  filter(!is.na(engagements))

# quick descriptive check
posts %>% 
  group_by(has_replied_comments) %>% 
  summarise(
    n = n(),
    median_engagements = median(engagements, na.rm = TRUE),
    median_log_engagements = median(log_engagements, na.rm = TRUE)
  )

# A tibble: 2 × 4
  has_replied_comments      n median_engagements median_log_engagements
  <lgl>                 <int>              <dbl>                  <dbl>
1 FALSE                118534                 11                   2.48
2 TRUE                  10447                 22                   3.14

These summary statistics show us that relatively few Threads posts have comments that have been replied to (only around 8%). However, on average, those posts tend to have significantly more engagement than posts with no comments that have been replied to.

This can be due to a variety of factors, including the possibility that posts with replied comments are simply higher quality or more engaging to begin with. To better isolate the relationship between replying to comments and engagement, we’ll use two complementary approaches that compare each profile to itself over time, Z-scores and fixed-effects regression.

Z-Score Analysis

A Z‑score analysis is a simple way to see how a given post performed relative to the account’s typical performance. Instead of comparing different accounts to each other, which can be unfair because some accounts naturally get more engagement, we compare each account to itself over time. A positive Z‑score means the post performed above that account’s baseline, and a negative Z‑score means it performed below.

For each account, we take the log of the number of engagements for every post and calculate the average and standard deviation for each account. Then, for every post, we compute a Z‑score: the post’s log‑engagements minus the account’s average log‑engagements, divided by that account’s standard deviation.

The resulting metric tells us how far above or below an account’s typical post engagement that specific post landed. This approach is more appropriate than using simple summary statistics because it focuses on within‑account differences rather than potentially comparing large accounts to small accounts.

# profile-level baseline on the log scale
profile_stats <- posts %>%
  group_by(profile_id) %>%
  summarise(
    mean_log_eng = mean(log_engagements, na.rm = TRUE),
    sd_log_eng = sd(log_engagements, na.rm = TRUE),
    n_posts = n()
  ) %>%
  filter(n_posts >= 3, sd_log_eng > 0)

posts_z <- posts %>%
  inner_join(profile_stats, by = "profile_id") %>%
  mutate(z_log_engagements = (log_engagements - mean_log_eng) / sd_log_eng)

# mean Z among posts that received any comments
z_any_comments <- posts_z %>%
  filter(has_any_comments) %>%
  group_by(has_replied_comments) %>%
  summarise(mean_z = mean(z_log_engagements, na.rm = TRUE))

z_any_comments

# A tibble: 2 × 2
  has_replied_comments  mean_z
  <lgl>                  <dbl>
1 FALSE                -0.0245
2 TRUE                  0.265 

These Z-scores tell us that, on average, posts with replied comments perform above a profile’s own typical engagement level and posts without replied-to comments sit slightly below the baseline. This is true even when we restrict the analysis to posts that received any comments at all.

Below are a couple plots that make the within‑profile lift easier to see. The first shows distributions of Z-scores whenever the post does and doesn’t have any comments that have been replied to. We control for whether or not the post has received any replies.

When posts have comments, the distribution of Z-scores is shifted to the right, indicating that they receive more engagement relative to the account’s own baseline engagement.

# 1) Z-score density by replied status (restrict to posts with any comments)
posts_z %>% 
  filter(has_any_comments) %>%
  ggplot(aes(x = z_log_engagements, fill = has_replied_comments)) +
  geom_density(alpha = 0.45) +
  labs(x = "Within-profile Z (log scale)", y = NULL, fill = "Replied to Comment",
       title = "Distribution of Performance by Replying to Comments",
       subtitle = "Posts with replied comments tend to perform better (higher Z-score)")

The second shows the distribution of per‑profile differences. Basically, it visualizes how much better or worse each profile tends to perform when it replies to comments versus when it doesn’t.

We can see that the distribution is centered at a point greater than 0, indicating that most accounts tend to perform better when they reply to comments. Roughly two thirds of accounts have a positive difference.

# build per-profile means (needed for difference plot)
profile_pair <- posts_z %>%
  group_by(profile_id, has_replied_comments) %>%
  summarise(mean_z = mean(z_log_engagements, na.rm = TRUE), .groups = 'drop') %>%
  tidyr::pivot_wider(names_from = has_replied_comments, values_from = mean_z)

# 3) Per-profile difference: mean Z (reply) - mean Z (no reply)
diff_df <- profile_pair %>%
  mutate(diff = `TRUE` - `FALSE`) 

share_pos <- mean(diff_df$diff > 0, na.rm = TRUE)
med_diff <- median(diff_df$diff, na.rm = TRUE)

ggplot(diff_df, aes(x = diff)) +
  geom_histogram(bins = 50, fill = '#2c7fb8', alpha = 0.8) +
  geom_vline(xintercept = 0, linetype = 2, color = 'grey50') +
  geom_vline(xintercept = med_diff, color = '#d95f0e') +
  labs(x = "Per-profile mean Z difference (reply - no reply)", y = NULL,
       title = "Most Profiles Perform Better When They Reply",
       subtitle = paste0("Median difference ", round(med_diff, 3), "; ",
                         round(100*share_pos, 1), "% of profiles > 0"))

Fixed Effects Regression

Next we’ll use fixed effects regression to estimate within‑profile models that compare each account to itself across posts. Fixed effects hold constant all time‑invariant differences across profiles—things like baseline audience size, niche, or brand strength—by comparing each profile to its own baseline.

Instead of asking whether accounts that reply to comments more get more engagement, which would mix large and small accounts, we’re calculating how engagement changes for each individual account when it replies to comments versus when it doesn’t. Modeling on the log scale also makes the model coefficients easy to read as approximate percent differences.

# FE on log engagements with controls, clustered by profile
fe_model <- feols(
  log_engagements ~ has_replied_comments + has_any_replies | profile_id,
  data = posts,
  cluster = "profile_id"
)
summary(fe_model)

OLS estimation, Dep. Var.: log_engagements
Observations: 128,981
Fixed-effects: profile_id: 13,836
Standard-errors: Clustered (profile_id) 
                         Estimate Std. Error t value  Pr(>|t|)    
has_replied_commentsTRUE 0.352520   0.015373 22.9305 < 2.2e-16 ***
has_any_repliesTRUE      1.284234   0.056973 22.5409 < 2.2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
RMSE: 1.06588     Adj. R2: 0.55732 
                Within R2: 0.011406

This is how we interpret the coefficients on the log scale: within the same profile, posts with replied comments see about exp(0.353) − 1 ≈ 42% higher engagements on average than posts without replied comments, holding constant whether the post received any replies.

Taken together with the Z‑score analysis, this suggests that engaging in the comments is associated with a significant lift in engagement.

Caveats

The main limitation of this analysis is that we don’t have timestamps for when engagements occurred relative to when comment replies were made. It’s possible that posts receiving high engagement early on are more likely to generate comments, and creators may be more motivated to reply to comments on posts that are already performing well. This means we can’t definitively say that replying to comments causes higher engagement. The relationship could go in the opposite direction, or both could be driven by other factors like content quality or timing. Despite this limitation, the consistent positive association we see across both fixed effects models and Z-score analyses suggests that comment engagement and post performance tend to move together.