Threads: Engagement and Replies

Overview

Buffer is launching a new feature called Community that will help creators quickly respond to comments on several different platforms. In this analysis we’ll look at the effect that replying to comments has on engagement.

What We Found

When we compare each Threads account to itself over time, posts with comments that have been replied to outperform the account’s own baseline by roughly 43%, on average. This result holds even when we restrict the sample to posts that received at least one comment.

An analysis of Z-scores also suggests that replying to comments has a positive impact on engagement. Posts with replied comments tend to sit above an account’s typical engagement level, while posts without replied comments sit slightly below.

The main takeaway is that replying in the comments is associated with a meaningful lift in engagement. This isn’t an unexpected result, but it’s nice to see it borne out in the data.

Data Collection

The SQL below returns around 84 thousand Threads posts sent in the past two years as well as the number of comments (replied and unreplied) that they received.

sql <- "
    select
        u.id as post_id
        , u.profile_id
        , u.user_id
        , u.sent_at
        , u.likes
        , u.replies
        , u.reposts
        , count(distinct c._id) as total_comments
        , count(distinct case when c.status = 'replied' then c._id end) as replied_comments
        , count(distinct case when c.status = 'unreplied' then c._id end) as unreplied_comments
    from dbt_buffer.publish_updates as u
    inner join dbt_buffer.community_comments as c
      on u.id = c.post_id
    where u.profile_service = 'threads'
      and u.sent_at is not null
    group by 1,2,3,4,5,6,7
"

# get data from BigQuery
posts <- bq_query(sql = sql)

Data Preparation

First we’ll construct a few core metrics and indicators, then filter to posts with non-missing engagement data. Here’s a quick descriptive check of median engagements by whether or not the post has any comments that have been replied to.

# construct core metrics and indicators
posts <- posts %>% 
  mutate(
    engagements = likes + replies + reposts,
    has_replied_comments = replied_comments > 0,
    has_any_comments = total_comments > 0,
    has_any_replies = replies > 0,
    log_engagements = log1p(engagements)
  ) %>% 
  filter(!is.na(engagements))

# quick descriptive check
posts %>% 
  group_by(has_replied_comments) %>% 
  summarise(
    n = n(),
    median_engagements = median(engagements, na.rm = TRUE),
    median_log_engagements = median(log_engagements, na.rm = TRUE)
  )

# A tibble: 2 × 4
  has_replied_comments     n median_engagements median_log_engagements
  <lgl>                <int>              <dbl>                  <dbl>
1 FALSE                77435                 10                   2.40
2 TRUE                  6113                 21                   3.09

These summary statistics show us that relatively few Threads posts have comments that have been replied to. However, those posts tend to have significantly more engagement than posts with no comments that have been replied to.

This can be due to a variety of factors, including the possibility that posts with replied comments are simply higher quality or more engaging to begin with. To better isolate the relationship between replying to comments and engagement, we’ll use two complementary approaches that compare each profile to itself over time.

Z-Score Analysis (within-profile)

We’ll standardize engagement within each profile to compare a post to that profile’s typical performance. Z-scores are computed on log(engagements + 1) to reduce the influence of viral outliers.

We apply the log transformation because the distribution of the number of engagements are long tailed. Most posts get modest numbers, while a smaller share gets very high engagement.

Taking logs stabilizes variance and makes the summaries and models less sensitive to those occasional viral posts. It also turns multiplicative differences into additive ones, which means effects can be interpreted as approximate percent changes.

# profile-level baseline on the log scale
profile_stats <- posts %>%
  group_by(profile_id) %>%
  summarise(
    mean_log_eng = mean(log_engagements, na.rm = TRUE),
    sd_log_eng = sd(log_engagements, na.rm = TRUE),
    n_posts = n()
  ) %>%
  filter(n_posts >= 3, sd_log_eng > 0)

posts_z <- posts %>%
  inner_join(profile_stats, by = "profile_id") %>%
  mutate(z_log_engagements = (log_engagements - mean_log_eng) / sd_log_eng)

# mean Z among posts that received any comments
z_any_comments <- posts_z %>%
  filter(has_any_comments) %>%
  group_by(has_replied_comments) %>%
  summarise(mean_z = mean(z_log_engagements, na.rm = TRUE))

z_any_comments

# A tibble: 2 × 2
  has_replied_comments  mean_z
  <lgl>                  <dbl>
1 FALSE                -0.0227
2 TRUE                  0.270 

On average, posts with replied comments perform above a profile’s own typical engagement level, while posts without replied comments sit slightly below their baseline. This is true even when we restrict the analysis to posts that received any comments at all.

In other words, replying to comments is associated with a significant lift in engagement, not just a shift in which posts get comments.

Below are a couple plots that make the within‑profile lift easier to see. The first shows distributions of Z-scores whenever the post does and doesn’t have any comments that have been replied to. We control for whether or not the post has received any replies.

When posts have comments, the distribution of Z-scores is shifted to the right, indicating that they receive more engagement relative to the account’s own baseline engagement.

# 1) Z-score density by replied status (restrict to posts with any comments)
posts_z %>% 
  filter(has_any_comments) %>%
  ggplot(aes(x = z_log_engagements, fill = has_replied_comments)) +
  geom_density(alpha = 0.45) +
  labs(x = "Within-profile Z (log scale)", y = NULL, fill = "Replied to Comment",
       title = "Distribution of Performance by Replying to Comments",
       subtitle = "Posts with replied comments tend to perform better (higher Z-score)")

The second shows the distribution of per‑profile differences. Basically, it visulaizes how much better or worse each profile tends to perform when it replies to comments versus when it doesn’t.

We can see that the distribution is centered at a point greater than 0 (around 0.35), indicating that most accounts tend to perform better when they reply to comments. Roughly two thirds of accounts have a positive difference.

# build per-profile means (needed for difference plot)
profile_pair <- posts_z %>%
  group_by(profile_id, has_replied_comments) %>%
  summarise(mean_z = mean(z_log_engagements, na.rm = TRUE), .groups = 'drop') %>%
  tidyr::pivot_wider(names_from = has_replied_comments, values_from = mean_z)

# 3) Per-profile difference: mean Z (reply) - mean Z (no reply)
diff_df <- profile_pair %>%
  mutate(diff = `TRUE` - `FALSE`) 

share_pos <- mean(diff_df$diff > 0, na.rm = TRUE)
med_diff <- median(diff_df$diff, na.rm = TRUE)

ggplot(diff_df, aes(x = diff)) +
  geom_histogram(bins = 50, fill = '#2c7fb8', alpha = 0.8) +
  geom_vline(xintercept = 0, linetype = 2, color = 'grey50') +
  geom_vline(xintercept = med_diff, color = '#d95f0e') +
  labs(x = "Per-profile mean Z difference (reply - no reply)", y = NULL,
       title = "Most Profiles Perform Better When They Reply",
       subtitle = paste0("Median difference ", round(med_diff, 3), "; ",
                         round(100*share_pos, 1), "% of profiles > 0"))

Fixed Effects Regression

Next we’ll use fixed effects regression to estimate within‑profile models that compare each account to itself across posts. Fixed effects hold constant all time‑invariant differences across profiles—things like baseline audience size, niche, or brand strength—by comparing each profile to its own baseline. Instead of asking whether accounts that reply more get more engagement (which would mix large and small accounts), we ask: when the same profile replies in comments on some posts and not on others, how does its engagement change? Modeling on the log scale also makes coefficients easy to read as approximate percent differences.

The outcome is log(engagements + 1). Our key variable is whether the post has replied comments. We also control for whether the post received any comments at all and whether it received any native replies, and include profile fixed effects.

# FE on log engagements with controls, clustered by profile
fe_model <- feols(
  log_engagements ~ has_replied_comments + has_any_replies | profile_id,
  data = posts,
  cluster = "profile_id"
)
summary(fe_model)

OLS estimation, Dep. Var.: log_engagements
Observations: 83,548
Fixed-effects: profile_id: 11,304
Standard-errors: Clustered (profile_id) 
                         Estimate Std. Error t value  Pr(>|t|)    
has_replied_commentsTRUE 0.355211   0.019320 18.3853 < 2.2e-16 ***
has_any_repliesTRUE      1.378484   0.065947 20.9030 < 2.2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
RMSE: 1.03665     Adj. R2: 0.559949
                Within R2: 0.012528

This is how we interpret the coefficients on the log scale: within the same profile, posts with replied comments see about exp(0.355) − 1 ≈ 43% higher engagements on average than posts without replied comments, holding constant whether the post received any replies.

Taken together with the Z‑score analysis, this suggests that engaging in the comments is associated with a significant lift in engagement.