Adversarial reasoning

I am a logical positivist by temperament and a skeptic by habit. A logical positivist wants claims tied to checks and identifiable premises. A premise is a statement that an argument starts from.

I want rules I can check and premises I can identify. I also want conclusions that either follow or do not. A conclusion follows when the rules carry the premises to it.

By that standard, Hegelian dialectic has always looked like rhetoric. Dialectic stages a disagreement, then tries to resolve its tension. To me, that turns vague disagreement into vaguer agreement.

That made it awkward to watch a version of dialectic help language models for a year. A token is a small unit of text. A large language model, or LLM, generates text one token at a time. At each step, it assigns probabilities to possible next tokens using the preceding context, then selects a continuation. This describes how text is generated; it does not mean the model can only repeat itself or that likely text is true.

Here is how I got cornered.

I set out to build an adversary, not a Hegelian

This section shows how my adversarial tool acquired Hegelian names.

I wanted an adversarial reasoning tool: a way to test a proposed answer by asking another reasoner to challenge it.

Multi-agent debate is a method in which several models examine the same problem. The motivation is that a model can become committed to a line of reasoning, while another perspective may notice what it missed. That is a reason to try debate, not a guarantee that models are independent or that one will catch another's error.

That intuition drives Du et al.'s multi-agent debate paper.

I built a Model Context Protocol (MCP) server in the Go programming language. It exposes tools to the application running a model, so the model can request their use. Mine used several reasoning strategies.

The important strategy ran three roles in a loop. The thesis was a proposed answer. An antithesis was a challenge generated by a different model. A synthesis was a model-generated attempt to reconcile useful parts of both. The separate role names do not establish that synthesis used a third distinct model. In the sketch below, the challenge receives the thesis, and synthesis receives both positions; the excerpt does not specify every prompt or how evidence is weighed.

The synthesis also carried a numerical Confidence score: a support rating attached to its candidate answer. The shown code does not tell us how that score was generated or checked against observed correctness, so 0.85 must not be read as an 85% chance of correctness. That was my configured target. The loop can stop early when the score meets the target; otherwise it stops when it has used cfg.MaxRounds. Reaching the round limit does not imply success.

Go — the dialectic loop, roughly

for round := 1; round <= cfg.MaxRounds; round++ {
    thesis     := generate(thesisProvider, problem, lastSynthesis)
    antithesis := challenge(antithesisProvider, thesis)   // a different model
    synthesis  := reconcile(synthesisProvider, thesis, antithesis)

    if synthesis.Confidence >= cfg.ConfidenceTarget {     // stop at 0.85
        break
    }
    lastSynthesis = synthesis
}

Here round is the current iteration, cfg.MaxRounds caps the number of iterations, and lastSynthesis carries the previous synthesis into the next thesis generation. cfg.ConfidenceTarget is the early-stop threshold; synthesis.Confidence is the score compared with it. This is a schematic loop, not a complete MCP implementation: provider setup, initialization, error handling, and returning a final result are not shown.

A different model or provider may offer different proposals, but that alone does not make its errors independent or give its answer an independent evidential basis. Models can share training material, assumptions, prompts, and context. Their disagreement can broaden a review; agreement is not independent evidence of correctness.

I meant this to be multi-agent debate, not Hegel. The model helping me build it suggested the name dialectic reasoning. It also supplied thesis, antithesis, and synthesis.

I kept those names because they described the structure exactly. That was my first annoyance. I built an adversary and accidentally inherited a philosopher.

The part I couldn't argue with

This section explains why the loop could free a model from a bad approach.

The tool did something I did not expect. In some cases, a model repeated a wrong approach with growing confidence, and a few debate rounds dislodged it. It did not happen every time. I saw it often enough to take seriously, but I give no measured success rate or controlled comparison of causes here.

Language models generate forward, and that can create a trap. Each next-token distribution depends on the text already in context. If the text so far supports a bad approach, that context can make continuations along that approach more likely. It does not make correction impossible: a model can produce “Wait, no” and revise itself. The basin of plausibility is my picture for a possible bias in continuation, not a literal force or a universal rule.

The antithesis prompt provides what I call licensed self-negation: it gives the model permission and an explicit task to challenge an answer already in context. That changes the prompt conditions; it does not add a formal inference rule, meaning a rule that derives one conclusion from premises. My working explanation is that this new context sometimes makes reversal easier to generate. The observation alone does not show that this is why the interventions helped; other effects of the prompts or process could matter.

Why "rubbish" and "works" are the same sentence

This section explains how I can reject Hegel's logic while keeping this procedure.

Here is the reconciliation that let a positivist sleep.

My objection was that dialectic is not, by itself, truth-preserving. A method is truth-preserving when true premises guarantee a true conclusion under its rules. Propositional logic works with statements that are true or false; valid inference rules preserve truth in that sense.

An LLM is doing something different when it generates text. It predicts continuations from context. I use “grammar” here as an analogy for the role sequence the prompt asks it to follow, not as a formal grammar or a truth test: a claim is followed by a challenge, and the challenge is followed by an attempted reconciliation.

That sequence tells the models what kind of conversational move to make. It does not show that the move is valid, that the reconciliation is true, or that dialectic works in general. My interpretation is narrower: a loose role structure can sometimes help a model reconsider its current answer, perhaps because it changes what continuations the prompt invites. That is a plausible account of my experience, not a demonstrated causal finding.

This conclusion concedes no metaphysics. Metaphysics makes claims about the fundamental structure of reality.

I need not believe that contradiction drives history. I also need not believe that Hegel discovered reality's structure.

I need only observe that a procedure helps a particular system. Then I can test that claim.

The metaphysics stays in the bin. The procedure earns its place at the bench.

But it only sometimes works — and the failures are the interesting part

This section explains why the loop fails, even when its debate sounds convincing.

The tool failed at least as often as it helped, in my experience. I give no run counts or formal evaluation protocol here, so read this as a personal assessment rather than a measured rate. I saw three recurring failure modes.

Collapse. A synthesis needs positions that make substantive, conflicting cases. If one position is weak, synthesis may simply pass through the stronger one. A weak antithesis can relaunder the thesis: it makes the original answer look reviewed without seriously testing it. A weak thesis can instead leave the antithesis unchallenged. And a strong argument is not necessarily a correct one; an articulate, confident error can win.

Convergence. Over repeated rounds, positions can appear to settle into agreement. In my use, this often happened after about five rounds; that is an observation from my sessions, not a general convergence rate or guarantee. Without an external check, the conversation may stabilize around mutual coherence. I call that an apparent consensus fixed point by analogy: no deterministic update rule or mathematical convergence result is established here.

Agreement, confidence, and accuracy are different. Agreement means that positions align; confidence is the score attached to a synthesis; accuracy means that the answer matches a correct result under an appropriate check. The sketch measures no agreement quantity, and it does not check correctness.

My stop condition amplified the risk. In another method, I had used a confidence score intended to express the solidity of one thought. That intention does not establish calibration: reported scores would need to match observed correctness frequencies. In this loop, I interpreted the synthesis score as rising alongside consensus. Agreement might inflate a support rating, but the code shown measures only synthesis.Confidence, not agreement. The score did not independently verify accuracy.

For a score c between 0 and 1 to have a probabilistic interpretation, outcomes would need to support that interpretation on the relevant task distribution. A calibration condition would be:

Pr(answer is correct | reported score near c) ≈ c

Here Pr means probability over answers checked independently on the relevant task distribution; observed correctness frequencies estimate it. The symbol c is a reported score, and “near” means a narrow chosen score range around it. For a finite range, calibration compares the correctness frequency with the mean reported score in that range, which is approximately c when the range is narrow.

If answers scoring near 0.85 were correct about 85% of the time in that setting, that would support calibration in that score range. It would still not certify any one answer or establish calibration at other scores. I provide no such calibration result for this tool here. The 0.85 threshold is only the configured stop rule.

Forced negation. A role can require disagreement even when its model has no defensible objection. The result may be a red herring: a distracting point that avoids the real issue. A model whose expectations favor the thesis may still manufacture a peripheral objection to satisfy the role.

Genuine disagreement goes for the throat; manufactured disagreement clears its throat. Once I learned that sound, I heard it everywhere.

Relaundering is worse than wasted work

This section shows how a failed debate can make an error look more authoritative.

Wasted tokens would be tolerable. The worse failure is manufactured authority. This means the process gives credibility to a claim it never truly tested.

When the loop absorbs a weak position, the surviving position does not merely persist. It now appears to have survived adversarial review.

A confident error can face a dutiful, limp antithesis. It then leaves with credentials from a process that never tested it.

I built an adversary to strip unearned confidence. Its failure mode prints that confidence instead.

Hegel gives this bug a useful name in my own framing: immanent negation, a challenge that arises from a thesis's own tensions rather than from an externally assigned role. I am using that as a practical interpretation, not claiming a complete or universally settled definition of Hegel's method.

An assigned antithesis can manufacture dissent when the model finds no supportable objection but the role demands one. A real criticism need not be an internal contradiction: it can identify a false premise, an unsupported assumption, missing evidence, a scope limit, or a counterexample. Conversely, a careful review can find no substantiated flaw. That answer is more useful than inventing one. Genuine flaws are not necessarily easy to test, either.

The dialectic that actually worked had a GPU in it

This section shows why evidence corrected a diagnosis that debate alone might have preserved.

I found the correction in a different project. I trained a relation-extraction model over several weeks. Such a model identifies stated relationships in text.

I also kept a running log of each session. That log became a working dialectic for a reason my tool did not.

The apparent diagnosis was straightforward: held-out F1 was about 1%, while training loss was about 0.5. F1 combines precision—the fraction of predicted relations judged correct—with recall—the fraction of required relations found. For one relation class, or counts pooled across classes, its standard formula is:

F1 = 2TP / (2TP + FP + FN)

Here TP is the number of correctly predicted relations, FP is the number of predictions not accepted as correct, and FN is the number of required relations the model missed. What counts as a correct match depends on the evaluation rule.

When the denominator is nonzero, F1 is a fraction between 0 and 1; it reaches 1 when there are no misses or spurious predictions. If all three counts are zero, the expression is undefined and the evaluation must specify a convention. A class-averaged F1 instead combines separately calculated class scores. This post does not specify whether its F1 was pooled or averaged across classes.

A percentage is a fraction per hundred: 1% is 0.01 and 4.4% is 0.044. F1 and these percentages have no physical units.

Held-out data was excluded from training. Training loss measures error under the objective used during training. Its numerical scale depends on that objective and on how values are combined across examples; this post does not specify either. A loss of 0.5 does not, by itself, mean 50% error or a 50% correctness probability. Overfitting is one possible pattern: performance on training examples is strong while performance on unseen examples is worse.

Those results made overfitting look like the cause, and more data looked like the fix. That was a crisp, confident thesis—but the evidence shown did not yet establish that the model extracted its training examples well.

The missing control was a comparison: decode and score the training split too. Decoding turns model outputs into predicted relations. The training split contains examples the model saw during training.

To interpret the split comparison, the two evaluations need comparable input preparation, interpretation of relation labels, decoding, prediction cutoffs, and scoring. Otherwise, a difference could reflect a changed evaluation procedure rather than split membership. Those are conditions for a fair comparison, not details established here about my historical run.

The comparison that changed the diagnosis

training loss        ≈ 0.50      # appeared to fit under the training objective
held-out  F1         ≈ 1%        # poor decoded extraction on unseen data
TRAIN     F1         ≈ 4.4%      # poor decoded extraction on data seen in training

The low training F1 undermined the specific story that the model extracted seen relations well and failed only on held-out examples. The two F1 values are both poor, but not equal; they differ by about 3.4 percentage points. That comparison weakens a generalization-only diagnosis and the claim that more data alone would fix the problem. It does not prove there was no overfitting or no train–held-out gap.

The control showed that failure was already present in the evaluated training-output path. The reported values do not identify its cause: insufficient task learning, an objective/metric mismatch, preprocessing or label mismatch, decoding, and scoring are all possibilities from these numbers alone. In my retrospective investigation, I located the fault in decoding rather than in the data. I do not describe the faulty component or a separate measurement of its repair here, so readers cannot derive that diagnosis from the displayed values.

My practical lesson from this project is to decode and evaluate the training split when diagnosing a suspected train/held-out gap, using a comparable pipeline. If training extraction is also poor, held-out novelty alone cannot explain the failure. This check is not an independent test of generalization and does not, by itself, locate the fault.

That measurement challenged a premise, not a confidence rating: the overfitting-only explanation assumed successful extraction on examples the model had seen, but decoded training output also scored poorly. In my practical use of the term, that is immanent negation: a check exposed a weakness in the thesis's own premise, rather than a model inventing an objection. The inference is limited by the validity of the decoding and scoring procedure.

The loss and F1 do not contradict one another mathematically. Loss describes fit to a training objective; decoded F1 describes whether predicted relations match the evaluation labels. They measure different stages and scales. A loss near 0.5 can coexist with poor decoded relation F1, so the mismatch calls for investigating the path from training objective to decoded output; it does not prove a successful extraction fit.

Each log entry followed an action on the world: I might run an evaluation, decode a split, or hit an out-of-memory error. Such actions supplied evidence that an argument could not settle merely by agreeing with itself.

My MCP loop placed arguments between actions, so it could drift toward agreement without testing it. Same dialectic; only one touched something that could say no.

What a skeptic gets to keep

This section states the limited conclusion that I, as a skeptic, can accept.

This is where I land.

Ungrounded dialectic can, in my experience, tend toward the most coherent agreement in the conversation and then mistake that agreement for truth. I was wrong only to call dialectic useless.

Add a referent—something outside the argument against which I can check a claim—and the same loose machinery can help constrain a stuck system with evidence. Connecting a claim to that check is grounding. A measurement or execution can test a particular claim. A second model can offer an additional critique, but it is not an external truth oracle: models may share data, assumptions, context, and systematic errors.

Grounding therefore does not guarantee truth. A test can reject a candidate only within the test's scope and subject to its measurement and scoring limits. The value is that a suitable, checkable result can expose a claim that the conversation itself has accepted.

Manufactured dissent returns empty. Genuine dissent returns with evidence.

The diagram is a structural picture, not proof or a universal recipe. It shows a design principle: make candidate claims answerable to a suitable check, where one exists. An external result can reject a candidate the debating models accept, but its force depends on whether the test actually measures the disputed claim.

That lets a positivist accept Hegelian machinery without converting. Negation has teeth only when it answers to something outside the conversation.

Debaters may settle into agreement; that does not mean their answers have converged in a mathematical sense or are true. A debate connected to a world can be better constrained by evidence, but no referent guarantees that every round reaches truth. The crucial question is what, if anything, can check the claim.

When a reliable external check exists, I want the process to touch it. Not every task has one. Hegel without a referent is rhetoric at its apparent fixed point. Hegel with a GPU is science with extra steps. I can live with that.