Peter Ching-Heng Pan: All, this is really a great honor to have you here today, and Dr. Jerzak, I will just ask Karl to introduce him. If this is your first time joining our series, I will briefly inform you that this is our fourth season since 2020. We have been greatly honored to have this opportunity to invite many top-notch scholars to join us, share their first-hand research, and discuss methodologies in political science and social sciences. Welcome aboard. Before we start, I would like to note that this initiative is sponsored by both National Chung Hsing University, a national university in Taiwan, and the University of Texas at Dallas. It has been a pleasure working with Dr. Ho and all the guests since 2020. Please stay tuned, as we still have a few more presentations and speeches in the coming weeks. I hope everyone enjoys today’s talk given by Dr. Jerzak.

Karl Ho: Right, perfect. Allow me to introduce our speaker. It is really my pleasure and honor to have Professor Connor Jerzak as our speaker tonight. Dr. Jerzak received his Master’s in statistics and PhD in government from Harvard University, where he was advised by Kosuke Imai, Gary King, and Xiang Zhou. Dr. Imai was our speaker in a previous series. During his graduate studies, he also worked as an intern at Adobe Research in San Jose, California, on the West Coast. Between 2021 and 2022, Dr. Jerzak completed a one-year postdoc with Adel Daoud and the AI and Global Development Lab at Linköping University in Sweden, while serving as a visiting scholar in the Program on Governance and Local Development at the University of Gothenburg. Since 2022, he has served as an assistant professor in the Department of Government at the University of Texas at Austin. His research has appeared in or is forthcoming in peer-reviewed journals in machine learning, economics, political science, and related fields. He will be teaching next spring at Harvard University. We are really pleased and privileged to have him speak to us at the Data Analytics Colloquium. His topic tonight is “Selecting Optimal Candidate Profiles in Adversarial Environments Using Conjoint Analysis.” Without further ado, allow me to introduce Professor Connor Jerzak—the floor is yours.

Connor Jerzak: Oh, thank you very much, Karl, and thank you, Peter; I appreciate you both for organizing this great series. Also, thank you to the audience for taking the time to be here with us today. Today, I am going to present some new research, and I am definitely looking forward to any questions or comments you might have. Because we have a bit of time today—around 1 hour and 50 minutes—I’ll first present the material, then we’ll dedicate the second half, about an hour or even an hour and 10 minutes, to a bit of a tutorial as well as any questions that you all might have; it would be great to discuss those. I can’t see the hands or the screens of other people in the room, so if you do have a question while we’re going through the presentation, definitely let me know and interrupt through the audio. Without further delay, let me begin by showing some data points. As this is a data analytics colloquium, let’s present some data on political candidates—something you might see in news reports or the public press, such as information on different political candidates; for example, in this case, Trump and Clinton. We might have survey data about the percent of voters or likely voters who think that one of the candidates would be better for certain issues.

In this case, 34% of voters stated that Clinton would be better at being honest and straightforward, whereas a different number said the same for Trump. Thus, we observe that these two candidates, Trump and Clinton, possess distinct attributes about which voters hold varying opinions; these attributes can be quite detailed, encompassing broad issues such as dealing with the economy, appointing Supreme Court justices, and serving as commander-in-chief—major policy areas that candidates might address. However, there can also be much more fine-grained policy distinctions between them, such as positions on individual income taxes, capital gains taxes, and corporate taxes. All these policy positions collectively define the candidates’ identities. The central question we will explore through this introduction is what renders a political candidate more preferable within the contexts of competition and institutional constraints.

As discussed in the slides regarding those favorability scores on various candidate attributes, these features are somehow integrated to make one candidate more appealing to a given voter and another less so. Therefore, we aim to understand this process, particularly amid two key aspects of the political landscape: competition between parties and institutional constraints, since politics does not occur in a vacuum. In this project, we will consider how to address the questions posed here. The overarching inquiries motivating us today concern these topics, though the methods we discuss are not limited to them but can apply to a broader class of phenomena involving competition and adversarial dynamics, which we will examine. It would be valuable to discuss with you how these methods might apply to your own work. With that overview in mind, let us delve into the details of approaching this question. One method political scholars have employed is the forced-choice conjoint design; some of you may be familiar with it, while others may not, given varying backgrounds in the group, but we will build it up step by step to include everyone. To provide more details on this research design under investigation today, it relies on a straightforward question—hopefully visible as I navigate these slides—in which we present potential voters, survey respondents, or similar participants with two different candidates: one example on the left and another on the right. These candidates feature attributes such as gender, partisanship, positions on various policies, ethnic background, and other personality characteristics. We display these two candidates along with their features, and on that basis the respondents indicate their preference.

Connor Jerzak: In this design, we randomize all features, determining whether a hypothetical candidate is male or female, Republican or Democrat, or favors cutting or raising taxes. We present respondents with numerous such candidates where every attribute has been randomized. The technical terminology describes this as independently randomizing each feature, often termed factors, assigning them to one of several options or levels. For instance, personality characteristics like compassionate versus strong leadership exemplify a factor, with the options representing its levels. This provides background on the terminology, which you may already know, explaining our review. An intriguing aspect is that we operate in a high-dimensional treatment regime. By contrast, many experiments involve simple treatments, such as randomizing a get-out-the-vote campaign or a survey manipulation to assess one effect against another—that is our usual approach. However, in conjoint experiments, we deal not with binary treatments but with a high-dimensional space where factors and levels combine to form complex treatments. This dimensionality will prove relevant to our discussion, so I highlight it now to keep it foremost as we examine the mathematical notation. We will break down the notation as thoroughly as possible; if questions remain, please inform me afterward, and we can resolve them. Within this research framework, allow me to outline some established methods, as they motivate our efforts to extend prior excellent work. A common analysis of conjoint data—perhaps one you have performed or encountered in assignments—involves estimating a key quantity called the average marginal component effect, abbreviated AMCE if unfamiliar. The core concept entails examining one factor at a time while averaging over others. Though the notation may appear daunting, it simplifies readily: essentially, the AMCE queries the average favorability—where Y denotes favorability—of a candidate with gender fixed as male versus female, averaged across all other feature possibilities, and we perform this calculation for each level of interest.

Connor Jerzak: According to the joint data-generating distribution, there is some terminology involved, but the basic idea is to compare all hypothetical male candidates against all hypothetical female candidates to determine which ones are chosen and which are not. This process generates the average marginal component effect, with an extension to interactions that is important and to which we will refer later. This approach has been developed in the literature, and we aim to engage with it, but I want to discuss it now to provide an overview if you are unfamiliar with it. Another development in the causality literature is the exploration of heterogeneous treatment effects, the notion that not everyone responds identically to the same candidate, leading to heterogeneity based on the respondent—whether a voter or someone else in a survey experiment. Different potential voters will respond variably, and in this context, considerable work has examined optimal actions for binary treatments given a person’s characteristics: if a certain voter type responds well to a treatment, administer it to maximize overall votes, for instance. These are among the advancements in this area. For those familiar with the literature, concepts like S-learner, T-learner, X-learner, tree-based methods, and causal forests are primarily developed for binary treatments and treatment effect heterogeneity. There has also been substantial work on individually optimal treatment rules, as well as recent analyses of heterogeneity with conjoint data. These are some tools for analyzing the conjoint data I mentioned earlier, each with strengths, and we seek to supplement them with additional methods, building on potential limitations of the current state of the art. One limitation is that average marginal component-wise effects examine only one factor at a time, such as gender, or perhaps two, like gender and ethnicity or gender and country of origin, but in reality, we may care about more complex interactions. Additionally, we average over potentially informative variability in the data. Another aspect we have sought but not found is the quantification of strategic considerations: politics does not occur in a vacuum but in a competitive environment where groups vie against each other for limited resources, a crucial part of the political narrative that we need to address further in this discussion. To contrast existing developments with our approach in this paper, our contributions focus not only on one or two factors at a time but actually on more comprehensive analyses.

Connor Jerzak: We aim to summarize the role of all the different factors in the experiment simultaneously, rather than examining just gender or just ethnicity, but considering all candidate features together. We will attempt this in an optimal manner to determine the ideal candidate profile for a specific voter type. Furthermore, instead of sidelining strategic considerations, we will prioritize them by developing methods to incorporate strategic dynamics explicitly, while also addressing institutional constraints and, crucially, propagating uncertainties throughout the process. This propagation is essential for statistical inference using the tools we are developing. If time permits at the end, I have slides detailing the computational tools we employ to handle these uncertainties, enabling you to apply them in your own research. Additionally, we might discuss various computational backends for this purpose, which I hope will prove informative. Overall, the optimal stochastic intervention framework we will explore today should usefully characterize the strategic dynamics inherent in conjoint data analysis. This provides a high-level overview of our approach; in the subsequent slides, we will delve into additional details to clarify our methodology, after which we can discuss applications. Now, let us proceed to the specifics motivating this project, beginning with a concept you may recognize: the stochastic intervention, which has appeared in the literature for several years. Allow me to explain it as clearly as possible. A stochastic intervention comprises several building blocks, the first being the expected value of the outcome under treatment T—where boldface denotes a vector of different political candidate attributes used in this conjoint experiment. The core idea is to assess the average outcome, such as candidate favorability, by assigning these treatment attributes under a new distribution. In the original experiment—perhaps conducted on MTurk or Facebook—a particular distribution was used to allocate features like the candidate’s gender. Here, however, we consider a counterfactual or hypothetical treatment distribution, denoted π, which differs from the experimental one. We employ this to estimate the average outcome under that alternative distribution. Thus, the goal is to understand what the average outcome would have been with a different treatment assignment method. There are several advantages to this approach, and one of the good things about it is its flexibility in modeling hypothetical scenarios.

Connor Jerzak: We are averaging across all treatment combinations. Here, the nice thing about this stochastic intervention is that we are not just examining one factor at a time, as we did with the AMCE that we just discussed; instead, we are looking at all aspects of the experiment simultaneously through this counterfactual probability distribution. Thus, this Pr(π) defines a useful model-based summary of the complex treatment process. These represent some strengths, but there are limitations as well; for instance, there is little guidance on choosing a good stochastic intervention given the wide range of possibilities available. To make this concrete, consider the stochastic intervention as a new distribution for allocating treatment—almost like a political strategy for deciding which candidates to upweight or downweight. That captures the intuition behind this stochastic intervention, sometimes referred to as an SI. Those are some of the strengths and limitations. What we do in this paper is discuss the notion of an optimal stochastic intervention. This equation—do not be intimidated by it—is exactly the same as the one above; the only difference is that we are now optimizing to find the distribution π that maximizes the average outcome under this new counterfactual treatment distribution. Hence, the concept involves taking the idea of upweighting or downweighting certain candidate attributes and selecting that procedure to maximize, for example, the favorability of a particular candidate; this strategy proves useful for summarizing how people respond to different kinds of candidates holistically, across all features rather than one at a time. This forms part of the discussion we will begin with and serves as a starting point. What I just described is useful initially, but without constraints, the optimal stochastic intervention reduces to simply finding the single best profile or candidate type. The issue arises because, without limitations on our search for that counterfactual weighting distribution, we could merely identify the best candidate globally. However, the treatments in this conjoint design are high-dimensional, resulting in far more treatment combinations than observations, which makes searching for the best profile among all possibilities infeasible. Therefore, we introduce some regularization; in your data science curriculum, you may have encountered Lasso or Ridge regression, and the idea here is similar but with a key difference.

Audience Member: Do we have any assumptions that all these factors or features are orthogonal? I think you are going to apply the regularization, but in the initial setup, no assumption is needed for these features, right?

Connor Jerzak: Correct. Thus, there are a few sorts of assumptions that we could make, and I will talk more about them, but the idea behind the counterfactual approach is foundational.

Connor Jerzak: The idea with the treatment distribution is that we could upweight and downweight features without limit, so there is no real bound on that process. We are going to discuss ways to introduce limitations to address the problem I just mentioned; however, in this framework up to now, we have not introduced any assumptions yet.

Audience Member: Okay, I’ll let you finish the regularization first. All right, then I may have more questions. All right, please go ahead.

Connor Jerzak: Sounds good. Definitely interrupt as I mentioned—since I can’t see anyone, just interject and we’ll discuss it. Therefore, the concept is that we need to impose constraints on the problem; otherwise, we would have to consider all different possibilities across these various candidate types, and there could potentially be millions or billions of such profiles given the high-dimensional treatment space. Thus, we introduce a bit of regularization, which is similar to ridge regression but with some differences. In ridge regression, we minimize the sum of squared errors while adding a penalty term to prevent the regression coefficients from becoming too large. Here, the objective is analogous in spirit, yet distinct in a few ways: first, we aim to maximize the average outcome under the new distribution, denoted as Q(π), which represents the average outcome under this new distribution—for instance, maximizing the favorability for a candidate based on their policy positions. Second, the penalty term applies not to coefficients in a linear model but to the counterfactual distribution π, which defines the counterfactual probability vector for assigning the different features hypothetically. We penalize deviations of this counterfactual probability vector from the data-generating probability vector P. For example, if there are two parties—Democrat and Republican—each assigned with 0.5 probability, then the entries in P would be 0.5 and 0.5; if strategically it makes sense to upweight Democratic candidates, the probabilities in π might differ, and we penalize elements in π that deviate too much from P. The intuition, which I highlighted in gray if you wish to refer to it, is that we seek a new distribution that maximizes the average outcome. We adopt this distributional approach to overcome the limitations of the AMCE, where we examined factors one at a time; now, these new probabilities are constrained to lie within a certain distance from the data-generating probabilities, as discussed analytically in the paper. The rationale is that when π diverges significantly from P—that is, when the counterfactual distributions are far from the data-generating ones—this substantially increases the estimation variance, which is why we incorporate the penalty to mitigate that variance. This regularized optimal stochastic intervention is what we will employ throughout the rest of the discussion.

For those familiar with the machine learning literature, this concept comes from statistical learning. The materials teach about shrinkage and regularization, with the goal of reducing estimation variance to improve the model. By shrinking some coefficients to zero, one can balance bias and variance, thus identifying the best model.

Audience Member: Great connection—this is a very smart way of doing that. Just to elaborate the similarities and differences a bit more: in regression, the model is for the conditional mean, whereas here it is for the policy distribution. Think of it like a strategy; pi is essentially a strategy. There’s definitely a similar flavor, although a few differences exist, and hopefully we can discuss them further. But continue.

Connor Jerzak: In the paper, there are several approaches to this, but the starting point I chose for intuition is a simple model. Although the paper presents a generalization, let’s begin more basically. First, consider a model for the outcome. We thus have two models: the counterfactual model for the strategy, plus a predictive model for the experimental outcome—specifically, support for a given candidate. We employ a linear model with interactions, interpreting it as modeling support via main features like the candidate’s party affiliation as Democrat or Republican, while incorporating all interactions among features. Here it is written out; despite abundant notation, it’s simpler than it appears. The model for Y(T)—where Y(T) denotes favorability toward a candidate with profile T—includes interaction effect terms, essentially summarizing relationships between all treatments and favorability responses. One contribution in the paper is deriving a closed-form variance-constrained optimal stochastic intervention. Given such an outcome model, we can derive the exact optimal stochastic intervention under regularization in closed form. This matters because it functions like a recipe, yielding the strategy from the model. Essentially, fit the model to conjoint data, then apply our derived recipe to condense complex interactions and extract the optimal strategy from experimental data, enhancing process understanding. The formula appears in the paper, involving linear algebra, with details potentially in the appendix. Our derived solution exhibits specific computational complexity: the big O term reflects a product, where L_d represents levels per factor. For instance, with five possibilities for a factor, L_d equals five. The core concept is that as the number of factors rises, complexity escalates multiplicatively.

As the number of factors and factor levels increases, the closed-form solution becomes slower to compute, though it remains reasonably fast and efficient; nonetheless, this provides intuition for what makes the problem easier or harder.

That covers our current progress, but some of you, especially those from a statistical background, may wonder how we approach inference in this setting.

What we have discussed thus far relates to estimation—specifically, how to estimate that optimal strategy, π_i^*—but the subsequent step involves understanding not only the point estimate but also the confidence intervals, which are crucial for research, the publication process, and communicating with policy officers or public officials.

To address this, we apply the Delta method, a technique for quantifying how uncertainties in the outcome model affect the policies under the optimal solution, in addition to the optimized expected outcome. I provide some details on this in the tutorial section if time permits, but the core idea is that the outcome model implies a certain optimal variance-constrained stochastic intervention; thus, to obtain standard errors, we propagate uncertainties from the model to the optimal using the Delta method, with further details to follow.

Hopefully, this intuition conveys how the approach develops; it serves as our initial foundation, though a few loose ends remain. One such issue is that insufficient regularization may yield estimates below zero or above one, presenting a constraint; another is the potential need to incorporate higher-order interaction terms or nonlinear outcome models, such as GLMs, neural networks, or various other forms. Therefore, the general method we discuss involves modeling the outcome and then computing the regularized optimal stochastic intervention via gradient ascent.

Essentially, rather than deriving a closed-form expression, we employ iterative optimization methods and continue to propagate uncertainties using the Delta method to derive standard errors—vital for the aforementioned reasons—leveraging specialized computational tools like automatic differentiation.

I aim to ensure we can briefly cover the tutorial material, which includes slides walking through how to implement something similar in your own projects to empower your research. In summary, we perform iterative optimization when a closed-form solution is unavailable for complex cases, while still obtaining standard errors through computational tools like automatic differentiation, which will be key for that discussion that I’ll leave to the tutorial; moreover, this approach is only really going to be limited by the outcome model.

The model parameter variance-covariance matrix, familiar to those with a statistics background, essentially summarizes how all the different parameters in the outcome model relate to each other in terms of covariances and variances. This variance-covariance matrix is the primary requirement for proceeding with our approach. Although it may seem abstract at present, we will discuss it further in the tutorial, where you will gain a deeper understanding of its details. Thus, I aim to equip you with these computational tools to advance your own work. At this midpoint in our discussion, we have covered optimization and various strategies, yet we have not addressed the most crucial element: the adversarial dynamics with which I began this talk. We all recognize that adversarial dynamics can be quite unattractive, but addressing them is essential. In most experiments, data are analyzed in an average-case manner, focusing on average marginal component effects or average marginal interaction effects, thereby averaging in some typical fashion. The core idea involves computing marginal quantities while averaging over sources of stochasticity. To simplify this potentially jargon-heavy concept, we examine one factor at a time and average over all others—that is the fundamental intuition. For instance, we can express the average-case optimal stochastic intervention using forced-choice data as follows. Until now, I have primarily discussed one candidate at a time, but in practice, we encounter two candidates competing electorally, or perhaps two parties in international relations engaged in conflict. In such scenarios, we can consider a maximization problem similar to the previous one, but now aimed at maximizing the probability that candidate A is selected over candidate B. Here, Y represents the utility of candidate A; if this utility exceeds that of candidate B, then candidate A prevails. In this average-case analysis, we optimize the strategy for A while averaging over all possible actions by B—this reflects the average-case framework prevalent in current experimental research. A few additional details: P again denotes the data-generating probabilities. If any aspects remain unclear or warrant further discussion, we can address them now or revisit these slides during the Q&A to thoroughly examine any questions you may have. This constitutes an average-case analysis focused solely on optimizing A, without considering B’s optimization. However, in reality, competition exists: group A vies against group B for scarce resources under constraints, as when party A competes with party B following primaries. Consequently, average-case analysis fails to account for the adversarial nature of the real world in this regard. Therefore, our primary contribution lies in contemplating how to introduce this notion of adversarial competition.

We apply this competition to conjoint data in a principled manner. What we introduce now is an approach with the following intuition: instead of merely examining Candidate A and attempting to enhance their favorability in isolation, we consider a more complex yet realistic and hopefully useful scenario where Candidate A seeks to maximize their vote share in a second round of voting while ensuring their candidate advances there. Meanwhile, Candidate B pursues the identical objective simultaneously. Consider this in terms of Republicans and Democrats: the Republican Party strives to select the optimal candidates on its side, as the Democrats do likewise. However, these two parties do not operate in isolation; they function within a primary system. If unfamiliar with the U.S. electoral system, it comprises two stages of competition. In the initial stage, parties engage in intra-party competition to secure the chance to compete in the subsequent stage against the opposing party. Thus, intra-party competition precedes inter-party competition. We quantify this dynamic as follows: here, we incorporate both a min and a max. Previously, we employed only a max; now, we integrate a min and a max. The concept is that Candidate A aims to win, but Candidate B does as well, formalizing this through a minimax framework where one seeks to maximize while the other minimizes in a sequential game-theoretic structure. The intuition is that Republicans optimize their candidate as Democrats optimize theirs, constrained by an institutional factor—the primary structure that organizes this competition. Together, this minimax setting and the institutional constraint provide structure to the remainder of the discussion. This forms the basis of the adversarial case for optimal causal intervention, featuring two parties contending against each other. We approximate a kind of Nash equilibrium via iterative optimization of A’s strategy and B’s strategy, inquiring how A can improve and subsequently how B can improve. We conduct this in an iterative manner; if familiar with machine learning, it resembles aspects of reinforcement learning with adversarial neural networks. We can still propagate all relevant uncertainties through automatic differentiation and the Delta method. Several computational elements occur behind the scenes; for instance, some targeted quantities require approximating expectations using Monte Carlo methods and optimizing through that process. Through the tools we hope to explore together, you will be able to perform such analyses, leveraging automatic differentiation and the Delta method, which are essential here—without them, progress would be limited. There are some nuances to this, and I aim to clarify everything. We are about 40 minutes into this presentation, but one key nuance is that we actually need two stages of competition to fully model the adversarial dynamics.

Connor Jerzak: This setup allows interesting adversarial dynamics to play out. If you’re interested in game theory, this makes total sense: the idea is that if Party A and Party B faced exactly the same constraints, they would just do the exact same thing if there’s a globally optimal strategy. If there really is a best approach involving certain kinds of policies, the two parties will basically converge to that best policy. Therefore, to have parties differentiate themselves, you actually need two stages of competition, such as a primary system or some level of inter-party competition followed by intra-party competition; otherwise, you get this global matching. I think that’s just an interesting tidbit, a connection with some concepts discussed in game theory but now linked to experimental analysis and design, and that’s something we can discuss further in the Q&A.

Let’s proceed. I’ve given you the big picture: where do we stand? We’ve talked about conjoint data and how important that is for studying things like how people respond to political candidates or other stimuli. I’m not sure about the background of everyone here, but if you’re in a business school or something similar, this could be relevant in the context of marketing research, where investigators want to know what makes an attractive product—it’s highly applicable there too. That’s the problem we started with, and then we considered how to analyze that data in a few different ways. However, we noted some limitations to those approaches, one being that everything is computed in a sort of average-case manner. Therefore, let’s look at everything together in some joint strategy, but that strategy has to take into account both one party and the other, both one actor and their counterpart—you know, both one country and the opposing country. Thus, we proposed a way of incorporating competition into our analysis of the conjoint data via variant-constraint adversarial interventions. We need some tools for that, and we’ll cover them in the tutorial part, but this is where we’ve gotten to, and hopefully you’re following along. If not, interject with questions, or we can talk about them in the Q&A. If the details—you know, details—really aren’t important, what’s crucial is the big picture, and then the details will gradually get filled in. This is the first thing we’ll discuss, but we do have simulations and other elements in the paper along those lines. If there’s time, I’m happy to talk through those things, since I think simulations, especially if you’re interested in pursuing academic research, can be a really valuable tool and helpful for some of you.

We can explore how that plays out in this context, where we are working with high-dimensional data; however, that discussion will be reserved for the end if time permits. What I wanted to share with you is really just the application of these methods in practice, specifically to better understand U.S. vote choice for president. Nevertheless, we could apply these methods that we have been developing not only to U.S. vote choice but also to other elections or settings beyond politics, which we can discuss during the Q&A. The data we will use come from the Ono and Burden experiment, designed to study the effect of gender on vote choice. The design was a forced-choice conjoint: respondents were presented with two candidates, Candidate A and Candidate B, and asked which one they would vote for.

Candidate features were randomized in exactly the same manner we discussed at the outset, and the randomized factors included policy position, gender, race, experience, party, and various other attributes incorporated into the experiment. The original study focused on the gender AMCE—recall that this acronym stands for average marginal component effect—as detailed in the initial paper. In contrast, we will investigate something slightly different by posing alternative questions. For instance, what would the optimal strategies be for various groups, such as all voters, Democratic voters, or Republican voters, across all features rather than just gender? We will also examine and compare the average stochastic intervention—remember, this represents the strategy in the average case, serving as a baseline—with the adversarial case, which I believe offers a particularly valuable perspective. Thus, we will contrast these two approaches, assess our findings, and evaluate whether one provides a more realistic understanding than the other; moreover, we will quantify this comparison, which I hope proves informative. First, we will present results on the optimized expected outcomes in different scenarios.

Specifically, we will examine the expected optimal Democratic vote share under the discovered strategy and compare it against the average marginal component effects. The concept here involves Democrats optimizing against an average-case opponent, meaning the Democratic candidate optimizes while the opponent does not. This scenario yields results like the following: (\hat{Q}(\hat{\pi}^*)) represents the expected Democratic vote share under this treatment allocation method, achieving approximately an 80% vote share, complete with confidence intervals. This outcome compares favorably to all the various average marginal component effects. Such is the result from our analysis in this average case. However, you might wonder—and I am certain you are considering this—why our estimated optimal vote share reaches 80% when real-world elections are typically much closer. The divergence between our estimates and observed reality arises because this analysis, by itself, fails to incorporate adversarial dynamics.

We have been focusing extensively today on the dynamics that make politics what it is. Now, we will compare the expected Democratic vote share not in the average case, but in the adversarial case, asking what happens when a Democrat optimizes against an adversarial opponent—this becomes our question of interest. The results shift dramatically: no longer is an 80% vote share attainable; instead, we obtain a vote share much closer to 51-52%, with the confidence band crossing 0.5, indicating that these outcomes align more closely with historical experience. We can verify this by examining observed data from real elections, where we have plotted the winning two-party vote share alongside our estimates for the optimal vote share under these strategies. In the average case, the result appears far outside historical norms, as seen in the upper plot, whereas in the adversarial case, that estimate is much closer to what we observe in the real world of politics—an aspect we consider important and can discuss further.

These represent the average outcomes under the different strategies. The final element I will share before concluding, perhaps followed by Q&A and a tutorial, is a preview of the strategies themselves. What I showed earlier was essentially the average vote share under these strategies, but you might wonder about the strategies in detail. Admittedly, there will be numerous results to present, so I want to summarize beforehand: the key insight is that adversarial case strategies qualitatively resonate with historical experience more than the average case strategy does.

That is the main takeaway, which I am providing upfront because there are many elements to examine, and it might initially feel overwhelming—yet I assure you, every point is comprehensible together. Here are some examples: in the average case, Republicans should upweight Black female candidates to increase their vote share, but we do not see such candidates emphasized as much; instead, other sorts are prioritized. In the adversarial case, we observe different kinds of candidates, more typically represented within Republican leadership, being upweighted. The concept is that primary competition dramatically alters the strategies inferred from the conjoint data and the analytic tools we are employing.

Thus, that forms the core takeaway. These illustrate what the strategies resemble: every point here is not an average marginal component effect but a probability under the new strategy of assigning a feature like this. In this instance, these are the results for the Democrats in the average case—all these are probabilities. Let us examine the top column, which concerns sex: we have male and female candidates. According to this average case analysis, male candidates should be upweighted; the probability of assigning a male feature should be elevated compared to female, which is why this point trends toward one. We have done this for all the other features.

The features in the experiment include age, family, race, experience, personality, party affiliation, competition, and others. We have conducted this analysis across all of them, obtaining new probabilities that represent strategies for each feature, revealing what Democrats should prioritize in the average case. This analysis can be performed not only among all voters—these are the results for that group—but also for subgroups, such as Democratic respondents, Republican respondents, and independent respondents. Thus, we can infer distinct strategies for these subgroups, comparing and contrasting their similarities and differences. This serves almost as a recipe book for policy positions likely to gain support among various subgroups, which is crucial for summarizing voter preferences.

That was the average case; however, certain aspects were unusual, such as the Republican strategy upweighting elements rarely seen among Republican candidates. This prompted the question of why, with the answer lying in the notion of strategic competition. Now, examining the Republican and Democratic strategies in this adversarial case yields very different outcomes that better corroborate observed strategic behavior. These are the strategies derived from the adversarial case; we could explore them further in Q&A, and I will summarize the results later. Broadly speaking, one advantage of framing the approach this way is that each strategy is embodied in a probability distribution.

For those familiar with probability theory, we can evaluate the likelihood of various outcomes under these distributions. Comparing the average and adversarial cases, we compute the log probabilities of observing real-world political candidates under each strategy. These log probabilities, drawn from actual candidates in real elections, are lower in the average case than in the adversarial one, indicating that strategies inferred adversarially align more closely with historical experience and observed behavior—a key insight for practical applications. Consequently, this approach enables comparison of actual strategic behavior against the inferred optimum, revealing when individuals or groups operate strategically versus less so. Having spoken for about an hour, I will now wrap up with some conclusions. First, by incorporating adversarial dynamics and integrating them with conjoint data, we find that neither Republicans nor Democrats hold a decisive electoral advantage in equilibrium using these tools; additionally, we observe that there is balance between the parties.

Connor Jerzak: Policy matching and deviation occur such that Republicans and Democrats align on some issues within the strategies we have been examining, yet they diverge on others; here are some examples for illustration. One intriguing open question involves the systematic analysis of how competition—the adversarial component—increases variance, or in other words, how competition amplifies the variance in political outcomes we observe, an area we are eager to explore further. In conclusion, the social, natural, and political worlds represent competitive and high-dimensional spaces, whereas the statistical summaries we employ are frequently unidimensional—focusing on one aspect at a time—and static, failing to incorporate the strategic dimension. Thus, our work aims to offer a pathway toward more holistic yet interpretable and comprehensible summaries of these complex effects and interactions, while integrating competitive dynamics. Numerous open questions remain concerning multiple players, such as outcomes in settings with more than two participants, which could prove highly relevant for electoral systems featuring additional major parties beyond the typical two, as well as for international relations where the number of actors often exceeds two by a considerable margin. Among other inquiries, for further details, here is a link; I have prepared a tutorial for the subsequent discussion, but I propose pausing here for questions. I am happy to share these slides, which are more practical-oriented should time permit delving into specifics, thereby enabling students and others to learn and apply these tools effectively. Nevertheless, let us halt for questions and discussion now, proceeding to the tutorial if time allows.

Audience Member: All right, thank you, Connor. We have a question, or rather a comment, from a student regarding marketing research examples. I believe you mentioned using conjoint analysis extensively, essentially examining new car configurations and feature combinations, which aligns with the traditional conjoint setup. Now, you introduce this method that enables finding the optimal combination across all features without concern for correlations, allowing the use of novel techniques to identify that optimum. Could you also discuss this in the marketing context? Before that, however, I think your mic may be generating some background noise—perhaps it is rubbing on something?

Connor Jerzak: Maybe it is me; I will try not to move. Is it… oh, the mic. Okay, yeah, go ahead. I will remain totally still for you all, but apologies for that. It is a great question. In the context of marketing research, several points come to mind: first, the approaches we are developing could be highly relevant, given the abundance of experiments in marketing research. Indeed, some colleagues at UT Austin in the business school are conducting causal studies.

We can conduct experiments to learn about the feature aspects that potential buyers respond to. For instance, in website design, we can determine which layout resonates best with users, be it one configuration or another. There are various ways to design a website in terms of color, dynamics, and other elements. The optimization we are working with here could readily be applied in such settings, where the objective is to improve outcomes in marketing or business contexts. The adversarial component would probably be less relevant there, insofar as it is usually not as prominent in marketing. However, it could apply when one product is analyzed against another. For example, when someone enters a grocery store to buy peanut butter, some jars appear one way and others differently, with varying ingredients and labelings. In that sense, there could be an adversarial component to the analysis, although I don’t think it has been explored. Our examples certainly haven’t addressed it, but if you have one in mind, feel free to email me, and we can examine it further. That’s a good question, and those are a few thoughts. Any other questions or thoughts? Okay.

Audience Member: Okay, Connor. Here’s a question from the audience. I’m not sure if it’s relevant, but it’s about whether there is a better way or a logical adjustment for respondents’ ability to rank the features of a profile. It might be off the topic you’re discussing, but do you have a response to that?

Connor Jerzak: Yeah, can you say the first part again?

Audience Member: Is there a way to distinguish or understand what a respondent is paying more attention to? Is that the question? There are different features of a profile and then some about respondents’ correspondence. I mean, for anyone to differentiate or try to weight the different features for respondents, and then maybe they try to come up with how different features have weight, right? But how can anyone more rationally calculate the different weight of each feature? I think the question comes from that kind of thing, and can we just adjust that? Otherwise, it could be merely an impression of certain things or certain candidates in a profile.

Connor Jerzak: Great question. The question is getting at this notion of which features matter and which don’t. We can address that in several different ways, and there was a good example of that. Thus, one thought is that the approach we discussed involves a two-step process: first modeling the outcome, then inferring certain consequences of that model based on the data. In this modeling context, not all features are weighted equally; favorability is affected more by some aspects than others, which impacts the optima we examine. Accordingly, in this case, we apply some regularization in the ML model. Therefore, we observe that all the new probabilities for personality traits are the same as the data-generating probabilities. The idea is that respondents don’t really care whether someone is honest or compassionate, at least in the data as analyzed—there’s no real responsiveness from voters along those traits. As a result, we see that the model assigns uniform probabilities to these traits.

Connor Jerzak: The new contextual distribution is actually the same as the old one, and this is due to the regularization we discussed, as well as the fact that there’s no response in the outcome model to these sorts of features. Thus, this approach incorporates the fact that respondents care more about some things and less about others. For example, respondents care a lot about security, abortion, and immigration, but they don’t really care much about personality—whether the candidate is compassionate, honest, or intelligent in terms of personality type; that’s just not really that important. Therefore, we’re accounting for that fact. However, there’s a deeper question: when designing your experiment, how many features do you include? Do you include just two factors like sex, age, and family, or more—five, ten? At a certain point, respondents are not going to be able to process all the information about these different attributes of the candidates. My sense is that there is a sweet spot in terms of how much richness we want in the number of features versus avoiding overburdening respondents by not having them incorporate too much information at once. I know there’s recent work on measurement conjoint designs which, of course, speaks to some of these issues as well. Hence, I think you raise good questions about how we adjust and try to deal with that in the context of the analyses discussed here. Great questions—any other thoughts or questions?

Audience Member: I have a couple more questions. Can you hear me? Yes, sorry, I was talking while I was muted. And my original question was: in the machine learning context, we actually can create Shapley values to attribute the probability contribution to each candidate or each feature, regardless of whether they are orthogonal. Thus, that is only looking at the way they can contribute to the prediction—say the whole model combination predicts 89%, so I can attribute, for example, the top candidate being 20% coming from the party ID, 15% coming from maybe an issue, maybe from other demographic variables, and so on. Therefore, that method is more generally looking at the prediction but providing some interpretable results. But your method is directly addressing the high-dimensional space and instead looking at just the probability contribution. Thus, this picture—this slide—is particularly interesting, for example, and this is exactly showing that such a decision could be high-dimensional. And in this case, personality is basically indifferent, right? So, personality has nothing to do with it, vis-à-vis other variables like gender and also issues. Therefore, do you want to compare these kinds of methods, and is this something you believe that machine learning models will find to get some more interpretability using this approach?

Connor Jerzak: No, interesting thought. I think we haven’t done a direct comparison with something like Shapley values or other sorts of summaries. I think that one of the motivations for doing this, as you mention, is to get some sort of interpretable output from the model. So, if we have some super complex model, how do we summarize it? And Shapley values might be one way, coefficients might be one way, but the approach here is, I think, nice because it is directly linked to the data.

Audience Member: The approach is interpretable with a probabilistic interpretation that incorporates dynamics. I agree that there is an interesting connection, and we can consider how it might be implemented in the paper. Additionally, Shapley values are harder to relate to real data, whereas our distribution-based method provides the probabilities directly.

Audience Member: Can you go back to the previous slides on the personality and adversarial cases? The personality effect is basically nothing. Can you provide a little more comparison, for example with expertise, where you can see a lot of indifference? Can you interpret that further—is it only on certain policies?

Connor Jerzak: Yes, exactly; it applies only to certain policies. To reiterate, the dashed line in these figures represents the data-generating probability. When points are far from the dashed line, the inferred strategy deviates significantly from the original data-generating distribution, whereas proximity indicates no difference in strategy. Thus, certain elements are distant from those dashed lines, while others remain close. The uniform level across these values arises because the original experimenters allocated treatments evenly, assigning all possibilities with equal probability. Given our data and incorporating all other factors, respondents do not seem to select candidates based on personality characteristics or their interactions with other elements. In our specified model, which includes main and interaction terms, observing everything at the data-generating probability suggests that personality does not influence choice probability, even when interacted with issues such as immigration, spending, abortion, or race. Consequently, since personality and its interactions do not affect the outcome, the strategy is set at the data-generating probability. That was a good question—great. Are there more questions from the US side or the Taiwan side?

Audience Member: And how computationally expensive is this, Connor? It looks like there’s a lot of computation involved.

Connor Jerzak: Yes, good question. We invested considerable effort on the computational side, and as mentioned, I am happy to send these slides, which contain some discussion on computation. The adversarial case analysis is by far the slowest among those discussed, with other cases completing in about 5 minutes, while the adversarial case requires 10 to 15 minutes. Hence, it operates on the order of minutes rather than hours or days. It can be even faster if the data are not particularly large. Should I discuss the tutorial now, or shall we continue with questions? I am not sure how practical the group is in terms of coding and related activities.

Audience Member: Yes, we have quite a diverse group though. However, I saw you have another 90 slides, and I would love to spend another two hours learning the code line by line. But let me check with Peter first to see how the Taiwan side is ready for a little more hands-on tutorial.

Audience Member: I think we’re already ready to go to the tutorial.

Connor Jerzak: Okay, sounds good.

Adel Daoud: Therefore, I thought it would be good to talk about this, since some of you might have a coding background while others might not; but through these slides, which I’m happy to send you, it will provide a recipe book for using the tools we’ve employed in this project to build the software underlying all the results we discussed today. It’s one thing to have an idea, but another to implement it in practice, and in your careers, learning the best computational tools could prove important for enabling you to accomplish tasks that are challenging for others, thereby giving you a leg up and empowering you in that way.

Hence, I want to discuss the tools we use in an effort to impart this knowledge so that you can apply it in your own work, consulting projects, or business- and policy-related endeavors. This tutorial should take around 20 minutes, after which we might have another Q&A before proceeding further; but what we do here is essentially discuss why we needed these tools and then explain how you can actually use and implement them in R. I’ve tried to be as clear as possible, working from the foundations upward. Thus, I won’t dwell on this because I think people get the message, but the basic point is that we have computational needs: what we were just describing involves optimizing complex objective functions and obtaining something called the Jacobian, which we need for the Delta method—that’s just something I’ll break down very simply in a minute.

The core idea is that, to optimize this complex objective function, we need an iterative approach. I’m not going to go through the details now—here are the details listed—but the key thing (and I’m happy to cover this in Q&A if desired) is that we use automatic differentiation to obtain gradients for this optimization. The fundamental concept is that we’re going to start with initial guesses for the parameters, then determine how we could adjust the parameters to best increase our objective function if we’re trying to maximize something, and then tweak the parameter estimates accordingly.

This approach is known as gradient ascent, and to obtain these gradients—which indicate the direction that would most increase our objective function—we use automatic differentiation. I’ll walk through how you can employ it in your own work with just one or two lines of code, hopefully empowering your research. Automatic differentiation is an algorithmic tool for computing gradients of things that might be hard to derive analytically; you might remember from math classes the chain rule, differentiation, and so on—we’re actually going to perform that exact process by tracking all the operations our code executes and computing the derivative of the entire function using the chain rule to relate everything, thereby allowing us to algorithmically compute the exact gradient of a really complex function even if we can’t write down the formula for that gradient.

Connor Jerzak: These computational tools address the first need we must meet, but there is a second need as well: even more complex generalizations of gradients to multi-dimensional inputs and outputs. The details are not crucial; I recognize that some of you may have a statistics background while others do not, but for those without one, the core idea is that to comprehend the variance of those optimal policies, we must grasp how elements relate to one another in a more multi-dimensional, holistic manner. I have included a few details on this slide, yet fundamentally, we are generalizing gradients to an even broader multi-dimensional space where the output of the function we are differentiating is itself multi-dimensional.

Thus, both the input and output are multi-dimensional, which is essential for computing the Jacobian matrix; this, in turn, is vital for determining the variance-covariance matrix of our estimates and ultimately for obtaining confidence intervals. In summary, this process is critical for deriving confidence intervals, and it would be exceedingly difficult without a computational tool like the one I will discuss shortly.

Specifically, the tool we will employ is available within R. I am unsure if R, Stata, or other computational platforms are your primary focus—I know many social scientists use R, so I hope some of you are familiar with it to keep us aligned—but the concepts we will cover have analogs in tools like Stata, though the notation may differ slightly. What we will explore is how to integrate R with Python: R is advantageous because many of us already know it, whereas Python excels due to its widespread adoption in the machine learning community, its robust support for automatic differentiation, and its capacity for efficient optimization to accelerate our code.

Without these tools, tasks that took us five minutes in this project might have required five months instead; thus, we optimized extensively. In the next few slides, if you allow, I will briefly outline how to integrate R with Python, emphasizing that in the current landscape, we are moving past identifying solely as R users or Python users. Increasingly, these computational tools—such as Stata in Python or R in Python—can be harmonized, enabling us to use Python within R entirely without leaving the R environment.

We will leverage Python tools via a package called reticulate, allowing us to develop what computer scientists term a backend by combining R with other libraries. This empowers us to conduct analyses that might otherwise be more challenging and time-consuming, and it underpins the packages we built for the tools under discussion today. To avoid prolonging this, I want to ensure time for additional questions, so I will merely provide an overview without delving deeply into details, given that backgrounds may vary among you when it comes to these topics.

Connor Jerzak: I’m not delving into the coding details right now, but in case some of you are interested, I have slides here explaining how to set this up on your computer. Many of you might have used R packages before; this process essentially involves loading the reticulate R package into your environment, setting up a computational environment with this line of code, and then importing Python libraries directly into R. Specifically, you import a package called JAX followed by a package called optax, and I’ve provided code at the bottom for installing these packages on your own machine.

These instructions work regardless of whether you are using Linux, Windows, or Macintosh, although there might be minor differences, they will function similarly across platforms. I should note that this particular code, “use_condaenv,” is specific to your computer, and I’m happy to discuss offline how to set it up precisely; however, the rest of the code remains identical. As you can see, this allows you to write JAX code directly.

JAX, a tool developed by Google, offers highly optimized utilities for linear algebra operations—such as those employed in this paper—and provides excellent support for automatic differentiation, which I highlighted as crucial. Among its capabilities, it facilitates tasks like matrix multiplication; thus, I’ve created an R function here that computes the product of matrices A and B using the JAX backend. The abbreviation “jnp” denotes JAX NumPy, enabling me to perform this matrix multiplication entirely within R, which is quite impressive since it eliminates the need to switch to Python or learn it anew—we accomplish everything in R, as demonstrated here.

Another key feature these libraries support is Just-in-Time compilation, where the concept involves enhancing the efficiency of a function like our matrix multiplication by compiling it into an optimally streamlined computational form. The “jit” operator essentially analyzes the function to identify the most efficient execution path; while the technical specifics are secondary for now, the primary objective is to eliminate redundancies and maximize optimization for potentially inefficient code.

Additional details on Just-in-Time compilation are included, but we will demonstrate later how it can accelerate processes dramatically. I’ve prepared some calculations here that you can experiment with offline if desired. Two key takeaways emerge: first, we are conducting JAX computations directly within R, empowering you to replicate this approach; second, upon reflection, I retract that slightly—one consideration is ensuring students use RStudio, as alternatives in plain R require managing multiple interpreters, given that various computers might have different Python installations in the backend. For instance, I personally use Anaconda, which differs somewhat from setups relying solely on R; therefore, I believe you mean using RStudio, correct? In that case, you gain interpreter selection options, and this guidance will prove helpful for some students in case they encounter setup variations.

I wanted to emphasize a few aspects in the next twenty minutes or so until we conclude. The first thing I will highlight is how there can be really massive speedups to base R. What I have done in the top code is less critical; what matters is the bottom part, where I have written R code for performing matrix multiplication, but I accomplish the same task with this JAX backend that we have been discussing a bit.

In terms of execution time, instead of taking twenty seconds to run through all this code, the matrix multiplication now takes much less time. This represents a significant speedup, which can make a substantial difference, especially as our projects increase in complexity. Thus, this serves as another example to provide you with experience and to demonstrate that it is actually simpler than it may seem if you lack familiarity with this approach. It requires some time to acclimate to the semantics, but if you persist and read the documentation, things usually become clear. The only difference is that, if you were actually in Python, this dollar sign would be replaced by a dot, whereas in R, dots are replaced by dollar signs, introducing a slight semantic variation.

Last, I have a few other thoughts about automatic differentiation. Remember that automatic differentiation was crucial for conducting optimization through what we call gradient ascent, but it was also essential for obtaining that Jacobian matrix, which illustrates how one element relates to everything else in the Delta method setting. We employed automatic differentiation in both of those cases, and here we have written a function that takes this matrix multiplication output and then simply sums it up.

This is my function, and now we are going to auto-differentiate to obtain the derivative of this function with respect to A, and then the derivative with respect to B. In some instances, you might be able to compute that analytically using your knowledge of calculus, but in many other cases, you might not—or might not want to—when seeking the gradients of some really complex function.

This tool is useful even for simple functions. Now we apply—we feed—this function, JAX fxn, into JAX’s gradient function, specifying the arguments with respect to which we want to differentiate. You see here we are differentiating with respect to the first and the second argument. In Python, just a word of caution: everything is zero-indexed, so to refer to the first and second positions, we use zero and one respectively, not one and two. Thus, in this highlighted line, we are computing the gradient of this function with respect to the first and second positions, and then we are compiling that function. The beautiful thing is we are doing this all in an efficient manner.

We evaluated that function in R, and in the next line of code, we used arrays A and B—essentially a matrix fed into the JAX backend. After calling it, we can access the gradient of the function with respect to A and B. The process was extremely fast and efficient, with very simple code to implement. This approach might interest you for your own work, such as building a library to solve problems you are addressing efficiently.

If folks are interested, I have one or two more slides. I wanted to discuss briefly using random sampling with JAX, which can be useful for simulations and related tasks. Here is code for random sampling within JAX. Given our limited time, I won’t dwell on details, but one key point is that you must specify a seed for every operation. In R, you set the seed once at the session’s start, and it persists.

With JAX, however, each random sampling operation requires a unique key, akin to setting a seed for every draw. Although it’s a little more involved, we can optimize the random sampling process using the compilation I just described. The speedups are more dramatic here: compiling reduces time from two seconds to nearly zero, making it highly powerful for your projects. Another takeaway is that linear algebra representations of arrays are so efficient with these backends that scaling the problem by a thousand often results in only a tenfold increase in computation time. We essentially replicated the earlier operation but multiplied everything by a thousand, yet the additional compiled time is less than ten times greater. Thus, efficient representations enable handling larger quantities with substantial speed benefits at relatively low cost. These costs include cognitive effort in learning the tools, which diminishes over time, and the minor effort to install JAX.

These examples illustrate how you might use such backends in your work, covering random sampling, automatic differentiation, and the just-in-time compiler. We observed that it’s not rocket science; although employed by machine learning researchers, there’s no reason it can’t apply to your projects. I’ve offered concrete examples, with more available online, but this captures the gist of the discussion. That covers the tutorial part essentially. I have more information and could expand on simulations or other aspects, but perhaps I should pause to check if anyone has questions or seeks additional thoughts on the coding or tutorial side. I’m happy to field any questions about this as well. Any questions?

Audience Member: I think this is a very interesting way of applying some machine learning techniques.

Audience Member: These techniques become particularly relevant in a conjoint setup, and this is the first time I have encountered this approach, where you navigate a high-dimensional space with such combinations; it is quite innovative. One can envision even more applications for it. I have never tried that myself, but we should incorporate it into our new curricula and other courses as well. Currently, we primarily focus on the R environment, utilizing both R and Python; this serves as an excellent example to demonstrate how to leverage both languages and their respective strengths. For instance, in Python, I have never used JAX, though I employ reticulate quite often. This represents a wonderful method for applying these techniques to modeling. Are there any questions from the Taiwan side or from USI? Students are wondering if the sample code can be shared. I checked your website, and there are some codes there, right, Connor?

Connor Jerzak: Yes, I can send you the code that goes through the JAX backend—take a look at that.

Audience Member: Very well. From my personal perspective for the students, it makes sense to focus on mastering one coding platform thoroughly. In this case, learn R really well and then incorporate Python when necessary through reticulate, given the significant cognitive cost of switching between them; such integration proves far more efficient mentally. That is one consideration for everyone.

Connor Jerzak: I totally agree.

Audience Member: We are advancing in that direction now, ensuring students achieve multilingual fluency. With that settled, are there any other questions about the code, the paper, or related matters? Hearing none, perhaps we should conclude here, Connor.

Audience: Let us conclude here. Thank you very much; it is late, but this session was truly informative, especially the eye-opening computational aspects. I greatly appreciate it. As a reminder, we will host another special presentation next week, so consider registering on our website at before you leave.