Instagram: Engagement and Replies

Overview

In this analysis we’ll look at whether replying to comments on Instagram posts is associated with higher engagement.

What We Found

When we compare each Instagram account to itself over time, posts with comments that have been replied to outperform the account’s own baseline number of engagements by roughly 21%, even after controlling for whether the post received any comments at all.

This is consistent with what we saw on LinkedIn and Threads, though the effect size is a bit smaller here. The coefficients tell us that when an account replies to comments on some posts and not on others, the posts with replies tend to see meaningfully higher engagement, holding constant all the other characteristics of that account.

Data Collection

The SQL below returns 712 thousand Instagram posts that have received at least one comment. Engagements use the up.engagements field when it’s available. Otherwise we fall back to summing the number of likes and comments that the post received.

sql <- "
  select
    up.id as post_id,
    up.profile_id,
    up.user_id,
    up.sent_at,
    up.likes,
    up.comments,
    up.shares,
    coalesce(up.engagements, up.likes + up.comments) as engagements,
    count(distinct c._id) as total_comments,
    count(distinct case when c.status = 'replied' then c._id end) as replied_comments,
    count(distinct case when c.status = 'unreplied' then c._id end) as unreplied_comments
  from dbt_buffer.publish_updates as up
  inner join dbt_buffer.community_comments as c
    on up.id = c.post_id
    and c.service_type = 'instagram'
  where up.profile_service = 'instagram'
    and up.sent_at >= '2025-01-01'
    and up.engagements > 0
  group by 1,2,3,4,5,6,7,8
"

# get data from BigQuery
posts <- bq_query(sql = sql)

Data Preparation

First we’ll construct core metrics and indicators, then filter out posts that didn’t receive any engagement. We’ll then calculate the median number of engagements that posts with and without replied-to comments received.

A quick note on the use of log transformations: the number of engagements are long‑tailed, meaning most posts get a small number of engagements while a few take off, so the log scale helps reduce variance and the influence of outliers, and makes effects easier to read as approximate percent changes.

posts <- posts %>% 
  mutate(
    has_replied_comments = replied_comments > 0,
    has_any_comments = total_comments > 0,
    log_engagements = log1p(engagements)
  ) %>% 
  filter(!is.na(engagements))

# quick descriptive check
posts %>% 
  group_by(has_replied_comments) %>% 
  summarise(
    n = n(),
    median_engagements = median(engagements, na.rm = TRUE),
    median_log_engagements = median(log_engagements, na.rm = TRUE)
  )

# A tibble: 2 × 4
  has_replied_comments      n median_engagements median_log_engagements
  <lgl>                 <int>              <dbl>                  <dbl>
1 FALSE                641547                 42                   3.76
2 TRUE                  70841                 61                   4.13

These summary statistics show that relatively few posts, only around 70 thousand out of 712 thousand, have comments that have been replied to. However, those posts seem to have achieved more engagement on average.

We should note that these are directional only. Posts with replied comments tend to show higher engagement on average, but we still need to compare each profile to itself over time to make a fair comparison.

Z-Score Analysis

A Z‑score analysis is a simple way to see how a given post performed relative to an account’s typical performance. Instead of comparing different accounts to each other, which can be unfair because some accounts naturally get more engagement, we compare each account to itself over time. A positive Z‑score means the post performed above that account’s baseline, and a negative Z‑score means it performed below.

For each account, we take the log of engagements for every post and calculate the average and standard deviation for each account. Then, for every post, we compute a Z‑score: the post’s log‑engagement minus the account’s average log‑engagement, divided by that account’s standard deviation. The resulting metric tells us how far above or below an account’s typical post engagement each specific post ended up.

# profile-level baseline on the log scale
profile_stats <- posts %>%
  group_by(profile_id) %>%
  summarise(
    mean_log_eng = mean(log_engagements, na.rm = TRUE),
    sd_log_eng = sd(log_engagements, na.rm = TRUE),
    n_posts = n()
  ) %>%
  filter(n_posts >= 3, sd_log_eng > 0)

posts_z <- posts %>%
  inner_join(profile_stats, by = "profile_id") %>%
  mutate(z_log_engagements = (log_engagements - mean_log_eng) / sd_log_eng)

# mean Z among posts that received any comments
z_any_comments <- posts_z %>%
  filter(has_any_comments) %>%
  group_by(has_replied_comments) %>%
  summarise(mean_z = mean(z_log_engagements, na.rm = TRUE))

z_any_comments

# A tibble: 2 × 2
  has_replied_comments  mean_z
  <lgl>                  <dbl>
1 FALSE                -0.0201
2 TRUE                  0.178 

These values tell us that, on average, posts with replied-to comments tend to sit above the account’s typical engagement level, while posts without replied-to comments sit slightly below.

Visualizing the Lift

The two plots below show the relative lift in engagement for posts with comments that have been replied to. The shift to the right of engagement for posts that have received comments confirms that when people reply to comments, the number of engagements tends to be slightly higher.

# Z-score density by replied status (restrict to posts with any comments)
posts_z %>% 
  filter(has_any_comments) %>%
  ggplot(aes(x = z_log_engagements, fill = has_replied_comments)) +
  geom_density(alpha = 0.45) +
  labs(x = "Within-profile Z (log scale)", y = NULL, fill = "Replied to Comment",
       title = "Distribution of Performance by Replying to Comments",
       subtitle = "Posts with replied comments tend to perform better (higher Z-score)")

The median difference in Z-scores for accounts is around 0.29, and 63% of accounts show a positive difference.

# Per-profile difference: mean Z (reply) - mean Z (no reply)
profile_pair <- posts_z %>%
  group_by(profile_id, has_replied_comments) %>%
  summarise(mean_z = mean(z_log_engagements, na.rm = TRUE), .groups = 'drop') %>%
  tidyr::pivot_wider(names_from = has_replied_comments, values_from = mean_z)

diff_df <- profile_pair %>% mutate(diff = `TRUE` - `FALSE`)

share_pos <- mean(diff_df$diff > 0, na.rm = TRUE)
med_diff <- median(diff_df$diff, na.rm = TRUE)

ggplot(diff_df, aes(x = diff)) +
  geom_histogram(bins = 50, fill = '#2c7fb8', alpha = 0.8) +
  geom_vline(xintercept = 0, linetype = 2, color = 'grey50') +
  geom_vline(xintercept = med_diff, color = '#d95f0e') +
  labs(x = "Per-profile mean Z difference (reply - no reply)", y = NULL,
       title = "Most Profiles Perform Better When They Reply",
       subtitle = paste0("Median difference ", round(med_diff, 3), "; ",
                         round(100*share_pos, 1), "% of profiles > 0"))

Fixed Effects Regression

Next we’ll use fixed effects regression to create within‑profile models that compare each account to itself across posts. Fixed effects hold constant all differences across profiles, things like audience size, niche, or brand strength, by comparing each profile to its own baseline.

Instead of asking whether accounts that reply more get more engagement (which would mix large and small accounts), we ask how engagement changes for each individual account when it replies to comments versus when it doesn’t. Modeling on the log scale also makes coefficients easy to read as approximate percent differences.

# FE on log engagements with controls, clustered by profile
fe_model <- feols(
  log_engagements ~ has_replied_comments + has_any_comments | profile_id,
  data = posts,
  cluster = "profile_id"
)

The variable 'has_any_commentsTRUE' has been removed because of collinearity (see $collin.var).

summary(fe_model)

OLS estimation, Dep. Var.: log_engagements
Observations: 712,388
Fixed-effects: profile_id: 67,734
Standard-errors: Clustered (profile_id) 
                         Estimate Std. Error t value  Pr(>|t|)    
has_replied_commentsTRUE 0.193742   0.005131 37.7561 < 2.2e-16 ***
... 1 variable was removed because of collinearity (has_any_commentsTRUE)
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
RMSE: 0.8771     Adj. R2: 0.774479
               Within R2: 0.003106

When we compare each Instagram account to itself over time, posts with comments that have been replied to outperform the account’s own baseline by roughly 21% (exp(0.194) - 1), even after controlling for whether the post received any comments at all.

This matches the pattern we saw on LinkedIn and Threads, though the effect size is a bit smaller here. The coefficients tell us that when an account replies to comments on some posts and not on others, the posts with replies tend to see meaningfully higher engagement, holding constant all the other characteristics of that account.

Caveats

The main limitation of this analysis is that we don’t have timestamps for when engagements occurred relative to when comment replies were made. It’s possible that posts that received a lot of engagement early on were more likely to generate comments, and creators may have been more motivated to reply to comments on posts that were already performing well.

This means we can’t quite say definitively that replying to comments causes higher engagement. The relationship could go in the opposite direction, or both could be driven by other factors like content quality or timing. Despite this limitation, the consistent positive association we see across both fixed effects models and Z-score analyses suggests that comment engagement and post performance tend to move together on Instagram.