About substantive change flagging

About substantive change flagging

This article explains what Kenii means when it flags a curriculum proposal as a
possible substantive change, why the flag exists, and why it is deliberately built
to inform a committee’s judgment rather than to make the decision for it. It is
about the reasoning and the mechanism. It does not tell you how to set the
thresholds or record a determination; those are tasks for the curriculum how-to
guides.

What “substantive change” means, and why software should be careful with it

Accreditors, and in the southern United States the Southern Association of
Colleges and Schools Commission on Colleges (SACSCOC), draw a line between routine
curriculum maintenance and a “substantive change”: a modification significant
enough that the institution may owe the accreditor notification or approval before
it takes effect. Whether a given change crosses that line is a matter of policy
interpretation. It depends on the accreditor’s current rules, on the specifics of
the program, and on facts that no course record fully captures.

That makes it a poor candidate for an automated verdict. A system that stamped a
proposal “this is a substantive change” would be claiming an authority it does not
have and cannot defend, and the first time it was wrong in either direction, by
raising a false alarm or by missing a real one, the institution would rightly stop
trusting it. Kenii therefore aims lower and more honestly. It does not decide. It
watches for the patterns that policies tend to care about and raises a hand so a
person looks. The name of the feature, substantive change flagging, is meant to
carry that limit: it flags, it does not rule.

The signals it watches

When a proposal is evaluated, Kenii computes a small set of signals from the
proposed change and compares each against a threshold. Four signals exist today,
and each is a rough proxy for a kind of change accreditor policy commonly treats
as significant.

• Credit-hour change. How far the proposed credit hours move from the current
  record, expressed as a percentage. A large swing in credit hours is a classic
  marker of a substantive change.
• New-content magnitude. The share of comparable fields on the proposal that
  differ from the current record. The code is explicit that this is a rough
  indicator of how much changed, not a measure of academic content, and that the
  institution, not the number, determines what counts as new content.
• Program-closure signal. Whether the proposal is a discontinuation. A closure
  is called out because it may warrant a teach-out plan and its own accreditation
  review.
• Delivery-method change. Whether the proposed delivery mode differs from the
  current record, since a shift in how a course is delivered can itself be a
  reportable change.

Each signal reports its computed value, the threshold it was measured against, and
a short note explaining what it looked at. Some signals decline to fire when there
is nothing meaningful to compare, a brand-new course has no prior credit hours to
measure against, for instance, and a record with no delivery-mode data yields no
delivery signal. The intent throughout is to surface a reason to look, with enough
context that a reviewer can judge whether the reason holds.

The thresholds belong to the institution

The line each signal is measured against is not fixed by Kenii. The system ships
with defaults, a credit-hour swing of twenty-five percent, a new-content share of
fifty percent, a delivery-share of fifty percent, and closure detection turned on,
but these are starting points, not rules handed down. An institution sets its own
thresholds as part of its curriculum validation rules, and those stored values
override the defaults. A college that reads its accreditor’s policy as stricter or
looser than the defaults imply can move the numbers to match its own reading.

The thresholds also carry a policy date, an “as of” stamp that records which
version of the governing policy the numbers were chosen to reflect. This is a
quiet but deliberate feature. Accreditor policy changes over time, and a threshold
that was right under one revision may be wrong under the next. Stamping the
thresholds with the policy they answer to gives an institution a way to know when
its configuration has fallen behind the rules it is meant to track. The default
stamp reflects the policy period the shipped numbers were drawn from, and it is
expected to be revisited whenever an institution reviews its settings against
current guidance.

Because the thresholds are institution-owned, two colleges running the same
software can legitimately flag different proposals. That is not a defect. It
reflects the reality that accreditation judgment is local, and it keeps the
authority where it belongs.

What the flag actually does

When a proposal is submitted, Kenii evaluates the signals. If any one of them has
crossed its threshold, the proposal is marked with a substantive-change flag; if
none has, no flag is applied, and a flag that no longer applies is removed. The
flag is a fact about the proposal, computed from its content, and it is visible to
the people reviewing it.

The flag does not block the proposal. Submission is governed by the institution’s
validation rules, which are a separate mechanism, and a flagged proposal proceeds
into review like any other. What the flag can do is inform the route. Because a
workflow can carry a rule that inserts an extra step when a particular flag is
present, an institution that wants flagged proposals to pass through an additional
review, an accreditation-liaison sign-off, say, can arrange exactly that. Whether
the flag changes the path is therefore the institution’s choice, expressed in its
workflow, not a behavior Kenii imposes.

The determination is where a human answers

The flag is only the prompt. The answer is recorded separately, as a
determination that a catalog manager enters against the proposal. The
determination is one of three deliberately plain values, yes, no, or unsure,
accompanied by a rationale and stamped with who recorded it and when. This is the
piece that carries institutional judgment, and its design says a great deal about
the feature’s philosophy.

The presence of “unsure” as a first-class answer is the clearest signal. A tool
built to decide would not offer it. A tool built to support a committee that is
still gathering facts, or that intends to ask the accreditor, needs it, because
“we have not concluded yet” is an honest and common state in curriculum
governance. The rationale field matters for the same reason: the value of the
record is not the yes-or-no but the reasoning behind it, preserved so that a later
audit, or a later committee, can see why the call was made. The flag and the
determination can even disagree, a proposal can be flagged and then determined
“no” with a documented reason, and that disagreement is the system working as
intended, not failing.

A note on where this can mislead

It is worth stating the limit plainly, because a flag carries a certain authority
simply by appearing. The signals are proxies. New-content magnitude counts changed
fields, not the intellectual weight of what changed; a one-word correction and a
wholesale redesign can move the same number of fields. A proposal can cross a
threshold and be entirely routine, and, more worryingly, a genuinely substantive
change can slip under every threshold, especially one whose significance lies in
program-level context that a single course record does not hold. The flag is a
reason to look, never evidence that the looking is done. An institution that
treats the absence of a flag as a clean bill of health has misread the tool. The
determination, and the human reasoning recorded with it, is the part that counts.

Scope and boundaries

This explanation covers what the flagging feature watches, how its thresholds are
owned by the institution, and why it stops at informing the committee. It does not
document the exact default numbers as a specification, the procedure for editing
validation rules, or the API through which determinations are recorded; those
belong to the curriculum how-to guides and to the
REST API reference. The way a flag can lengthen a
proposal’s route connects to the review workflow, and the wider reason proposals
are kept out of the live catalog until they are approved is covered in
Why proposals stay isolated until approval.