Keep the human override alive in AI-assisted cash targeting
Cashy recommends which displaced households receive cash. A caseworker decides. Design how that caseworker keeps saying no when the AI is wrong, and how the institution can see when they stop.
How UNHCR targets cash assistance
UNHCR gives cash to forcibly displaced households so they can cover basic needs such as food, rent and medicine. Funding never covers every household that needs help, so each operation has to target: decide who receives cash, and in what order. In the UNHCR operation in Latin America behind this challenge, targeting follows five steps.
Interview
Registration staff interview the household: who they are, documents, health, housing, basic needs and how they cope. The interviewer also records their own view of the case.
Scorecard
An expert-designed Scorecard turns the answers into a final score and a vulnerability category, and runs administrative checks (asylum procedure, stated intentions, duplicate registration).
AI recommendation
Cashy, a UNHCR prototype, predicts the score and category and recommends include or exclude, with a written reasoning beside it.
Decision
A caseworker reviews the case and accepts or overrides the recommendation. The household is not present and cannot contest it.
Prioritized cash distribution
Eligible households receive cash in order of priority, as far as funds allow. Others go to a waiting list or are excluded.
-
No feedback
The caseworker never learns whether a household was correctly included or excluded, so experience cannot correct the judgment.
-
No contest
The household is absent from the decision, so the caseworker is the only check on the recommendation.
-
Rationed assistance
Under a fixed budget, every wrongful inclusion is someone else's exclusion.
What Cashy is
Cashy is a decision-support prototype built in the operation's data-collection platform, KoboToolbox. An answer engine (an AutoML tabular model, AutoGluon, trained on about 19,900 historical Scorecard records) produces the score, the vulnerability category and the include-or-exclude recommendation. A reasoning engine (a large language model given the same anonymized household data) writes the case summary shown beside that answer.
The challenge: two questions
Solutions may address either question. The second is the core of the challenge: a better model helps, but it will still be wrong sometimes, and a caseworker has to catch it when it is.
Can we predict better which households should receive cash?
How do we make sure caseworkers keep overriding the AI when its advice is wrong, and that the institution can see when they stop?
Starting points for question 2, not limits
Keep Responsible-AI statements off the decision screen
Put them in training and documentation instead.
Run an override audit before deployment
Measure correct override on already-decided cases before a system reaches a consequential decision.
Record override in production
So a falling rate is visible to a named owner over time.
What you can build
This is a 48-hour hackathon. Design with an innovation mindset, but stay on the two questions, above all the second: what changes what a caseworker does with an AI recommendation, and how can the institution see it. Solutions may combine any of the following.
An instrumented targeting prototype
A Cashy-like model on the synthetic sample (score and category reproduction, eligibility prediction with calibrated uncertainty, per-case feature attribution) wrapped in a review screen that shows reasoning and answer as separate outputs, requires each to be endorsed on its own, and logs agreement, override, the reasons given and the reasoning-answer rating gap for every assessment.
An override audit toolkit
A reusable protocol and code to select adjudicated cases, collect the AI-assisted decision blind, reveal the determination, and score override. Returns the appropriate-reliance decomposition and end-to-end accuracy with confidence intervals, handles clustering by participant, and tells an operation how many caseworkers and cases it needs.
Decision-screen designs and an experiment plan
Interface variants that keep Responsible-AI statements off the decision screen, separate the Process-of-Thought and Process-Guarantees panels, or test cognitive-forcing, uncertainty or verifiable case-level cues, each with a pre-specified confirmatory contrast and minimum detectable effect.
A production override monitor
A Power BI compatible dashboard that records override in production, tracks correct-override and over-reliance rates over time, by office, vulnerability band and error direction, and raises a falling override rate as an event a named owner must explain.
Behavior-first governance instruments
Methods that combine principle ratings, model cards and self-assessment checklists with behavioral evidence, so a system is judged on how it is used and not only on how it is perceived. Includes early warning signs, such as unprompted praise of the system in open comments.
Research designs for the follow-up
An independent expert panel rating each answer and reasoning against the case file, a second coder for the comments, a participatory study with displaced households who never see the decision made about them, or a replication in another operation or domain.
Data
A synthetic sample generated from the historical Scorecard records on which Cashy was trained. It contains no real household. It keeps the Scorecard's structure, the values each variable can take and the score arithmetic, so you can build and test a Cashy-like model on it, but it does not reproduce the operation's real distribution. One row is one household assessed at a targeting interview: 1,900 rows, 26 columns.
Annex I: data dictionary
Read this before you open the file. Column names are the operation's own, some in Spanish; the dictionary gives an English name for each and translates the Spanish values. Three terms matter most.
FinalScoreFinal score
The Scorecard total: Demographics_Score + NeedsandCoping_Score, from 0 to about 81. Higher means more vulnerable. It is banded into the vulnerability category. It measures need; on its own it does not decide who receives cash.
EligibilityTargetEligibility target
The ground truth for this challenge: the operation's final recorded decision, INCLUSION or EXCLUSION. It is funding-driven: how many households are included depends on the money available in each period, and administrative checks can exclude a household whatever its score. That is why it overlaps with the final score and is hard to predict. It is the institution's standard, not a measure of a household's need.
Interviewer perception
The interviewer's own view of the household's situation, written in the registration record during the interview. Cashy's reasoning engine read it and caseworkers saw it on the review screen. It is free text that could identify a household, so it is not in the synthetic sample. If your design needs it, simulate it and say so.
| Column in file | English name | Meaning | Values in the sample (count) |
|---|---|---|---|
| Outcomes: the target. Do not use as model inputs | |||
EligibilityTarget | Eligibility target | Ground truth. Binary decision derived from Elegibilidad: the two "Elegible" values are INCLUSION, all others EXCLUSION. Funding-driven (see above) | INCLUSION (427), EXCLUSION (1,473) |
Elegibilidad | Eligibility status | Detailed decision recorded by the operation (six values). It encodes the target | Elegible, eligible (414); Elegible por Proceso Acelerado, eligible via accelerated process (13); Lista de Reserva, waiting list (451); No Elegible, not eligible (932); No Elegible por Intenciones, not eligible: stated intentions (52); No Elegible por Duplicidad, not eligible: duplicate registration (38) |
| Aggregated scores | |||
FinalScore | Final score | Scorecard total: Demographics_Score + NeedsandCoping_Score (differences under 1.5 points are rounding) | 0 to 81.1 (mean 27.7) |
Demographics_Score | Demographics score | Sum of the demographics block | 0 to 43.8 (mean 8.1) |
NeedsandCoping_Score | Needs and coping score | Sum of the needs and coping block | 0 to 50.0 (mean 20.1) |
Vulnerability_Score | Vulnerability index | A monotonic transform of FinalScore | 2.0 to 4.34 (mean 2.80) |
Vulnerability_Category | Vulnerability category | Band of FinalScore: Low up to about 24, Moderate about 25 to 37, High about 31 to 52, Severe above 52 (bands overlap slightly) | Vulnerabilidad Baja, low (775); Moderada, moderate (394); Elevada, high (612); Severa, severe (119) |
| Scorecard factor scores (higher = more vulnerable) | |||
Demographics.HH.Head | Head of household | Profile of the head of household | 1.0, 1.60, 2.07, 2.70 |
Demographics.Language | Language barrier | Language barrier | 1.0, 1.53, 2.05 |
Demographics.Profiles | Specific needs | Specific-needs profiles in the household | 1.0, 1.25, 2.58, 3.25 |
Demographics.Documentation | Documentation | Documentation status | 1.0, 1.56, 2.13 |
Needs_and_Coping.BasicNeeds | Basic needs | Unmet basic needs (food, water, hygiene) | 1.0, 1.78 |
Needs_and_Coping.Housing | Housing | Housing stability, from stable to street or night-only shelter | 1.0, 1.50, 2.12, 2.82 |
Needs_and_Coping.Neg.mechanism | Negative coping | Negative coping mechanisms | 1.0, 1.79, 2.58 |
Needs_and_Coping.Dependency | Dependency | Dependency burden | 1.0, 1.04, 1.74, 2.54 |
| Household attributes | |||
NumIntegrantes | Household size | Number of household members | 1 to 11; 925 single-person households |
dependencyCategory | Dependency category | Dependency ratio category | low (1,148), average (610), complete (94), high (48) |
FemaleHeadedHousehold | Sex of household head | Whether a woman or a man heads the household | jefatura_femenina, female-headed (484); jefatura_masculina, male-headed (491); blank for single-person households (925) |
CuidadorSolo | Sole carer | The head is the only carer of dependents | si, yes (373); no (602); blank for single-person households (925) |
HablaEspanol | Speaks Spanish | Whether at least one adult speaks Spanish | espanol_uno_mas_adultos, one or more adults (1,622); espanol_ningun_adulto, no adult (278) |
Analfabeta_si | Adult illiteracy | Whether any adult in the household cannot read | adultos_ninguno_analfabeta, no adult (1,664); adultos_uno_mas_analfabeta, one or more adults (236) |
| Administrative flags (0 or ±500; can override the score) | |||
ScoreCOMAR_PIL | Asylum procedure flag | Status in the national asylum procedure (COMAR) | 0 (771), -500 (155), 500 (14); blank where not applicable (960) |
ScoreIntenciones | Intentions flag | Stated intentions incompatible with the programme | 0 (1,844), -500 (56) |
ScoreDuplicidad | Duplicate flag | Duplicate registration | 0 (1,852), -500 (48) |
| Interview record | |||
month | Month | Month of the targeting interview (YYYY-MM) | 2023-06 to 2024-07 |
OficinaACNUR | UNHCR field office | Anonymized code of the office that ran the interview (ACNUR is UNHCR in Spanish) | sotap (1,278), fupal (190), foten (186), pcr_cdmx (127), futij (47), fomon (36), fusal (15); blank (21) |
| none | Interviewer perception | Interviewer's own view of the case, recorded in the registration record | Not in S8 (free text, withheld) |
In the synthetic sample the eligibility target is almost unrelated to the final score: every vulnerability band contains both outcomes. This exaggerates what the real, funding-driven data showed. Use the sample to build, instrument and test the workflow, not to claim model performance against the 67.6% reported for the real held-out set.
Missing values and score arithmetic
Blanks mean "not applicable". The 925 single-person households (NumIntegrantes = 1) have blank FemaleHeadedHousehold and CuidadorSolo, because the questions are not asked of them. ScoreCOMAR_PIL is blank for 960 rows where the procedure check did not apply. OficinaACNUR is blank for 21 rows. Treat these as structural, not as errors to impute away.
Score arithmetic. FinalScore equals Demographics_Score plus NeedsandCoping_Score. Vulnerability_Score is a monotonic transform of FinalScore, and Vulnerability_Category a banding of it. Each factor score takes three or four discrete values between 1.0 and 3.25.
What a caseworker sees. The fields in this file are the fields Cashy's answer engine received. A row rendered as a registration record, together with Cashy's reasoning and answer, is what a caseworker sees on the review screen, so the sample is also raw material for building review-screen stimuli.
Other released materials and the full brief
Where released for the hackathon, the Data folder also contains the participant briefing (S1), the codebook of the open comments (S2), the analysis code (S3), the de-identified assessment-level data (S4, 155 assessments from 31 participants) and principle-ratings data (S5), the model card (S6), the Cashy AutoML code (S7) and the redacted household narratives (S9). Household records and free-text comments are withheld because they could identify participants or households.
Full challenge brief: Data & Instructions (DOCX). Hackathon FAQ: FAQ (DOCX).
Experiment materials
An earlier randomized experiment in the operation tested how 31 caseworkers used Cashy. Annex II gives its instruments and reference results so teams can rebuild the workflow, run an override audit or design a review screen, and compare their measurements. Open only what you need.
What the experiment found
Each caseworker assessed five real, already-decided households in Cashy, with or without Responsible-AI statements on the decision screen: a Process-of-Thought panel describing how the system reasons in general and a Process-Guarantees panel promising transparency, accountability and responsibility. Neither panel said anything about the household under review.
Where Cashy contradicted the operation's own decision, caseworkers without the statements overrode it on 96.2% of assessments; caseworkers with the statements, on 66.7% (risk difference 29.5 points, 95% CI 11.9 to 47.0, p = .007). End-to-end decision accuracy fell from 83.1% to 68.9%. Eight of the twelve flawed recommendations accepted under the statements would have withheld assistance from an eligible household. Nobody in either arm overrode a sound recommendation. Post-session ratings against the UN system's AI principles did not register the loss: a statement about the system's conduct worked as a cue about its competence, and only behavior revealed the loss of oversight.
At these rates, every 1,000 discordant cases would carry about 295 more decisions against the operation's own standard.
Reference results: the five households and the reliance decomposition
The five households
Five real, already-decided households were selected to vary on vulnerability and on agreement with the Scorecard. Two are concordant (C4, C5), three discordant (C1, C2, C3). C3 was chosen as a hard case: the Scorecard categorized the household as highly vulnerable but excluded it on criteria unrelated to need, so it was excluded from the confirmatory set (C1 and C2).
| Case | Household profile | Relation (Cashy minus Scorecard) | Cashy score / recommendation | Scorecard score / determination |
|---|---|---|---|---|
C1 | Single adult, collective shelter, several basic needs unmet, one negative coping mechanism | Discordant, +29.38, Exclusion to Inclusion | 43.09 / Inclusion | 13.71 / Exclusion |
C2 | Sole carer with infant, both with serious medical conditions, stable housing, incomplete documentation | Discordant, -9.88, Inclusion to Exclusion | 25.52 / Exclusion | 35.40 / Inclusion |
C3 | Two-parent household, young child with serious medical condition, night-only shelter, high-risk coping | Discordant (contested), +14.47, Exclusion to Inclusion | 55.54 / Inclusion | 41.07 / Exclusion |
C4 | Single adult, collective shelter, all basic needs met, no health condition | Concordant, -8.59, no change | 15.79 / Exclusion | 24.38 / Exclusion |
C5 | Single adult, disability and medical condition preventing work, food/water/hygiene needs unmet | Concordant, 0.00, no change | 65.57 / Inclusion | 65.57 / Inclusion |
Appropriate-reliance decomposition by arm
Percentages with Wilson 95% CIs. Confirmatory columns use the 62 assessments of C1 and C2; concordant columns the 62 of C4 and C5. An override audit toolkit should reproduce these from the released assessment-level data.
| Condition | Correct override | Over-reliance | Correct acceptance | Under-reliance | End-to-end accuracy |
|---|---|---|---|---|---|
| Plain AI (13 caseworkers) | 25/26 = 96.2% [81.1, 99.3] | 1/26 = 3.8% [0.7, 18.9] | 26/26 = 100% [87.1, 100] | 0/26 = 0.0% [0.0, 12.9] | 54/65 = 83.1% [72.2, 90.3] |
| Responsible-AI statements (18 caseworkers) | 24/36 = 66.7% [50.3, 79.8] | 12/36 = 33.3% [20.2, 49.7] | 36/36 = 100% [90.4, 100] | 0/36 = 0.0% [0.0, 9.6] | 62/90 = 68.9% [58.7, 77.5] |
Participant-level risk difference in correct override: 29.5 percentage points (95% CI 11.9 to 47.0; Mann-Whitney p = .007; GEE OR = 0.080 [0.010, 0.639]). Ratings of identical advice: top-box correctness rose from 7.7% to 27.8% (Holm p = .005) while mean correctness (3.26 against 3.52) and appropriateness did not differ; the reasoning-answer gap on the confirmatory households was +0.81 (plain) against +0.64 (statements), p = .65. Principle ratings by arm (28 linked participants): accountability 3.00 against 3.61 (p = .047, uncorrected), transparency 3.30 against 3.94 (p = .078), fairness 3.20 against 3.28; none survives Holm correction. Minimum detectable effect at 80% power: d = 1.06 for the confirmatory contrast, d = 1.15 for the post-session instrument.
The Responsible-AI statements shown in the statements arm
Process-of-Thought. How was the general analysis done?
- Data collection: the most important data were taken from each form: who they are, what documents they hold, their health and disabilities, where they live, whether they can cover their basic needs, how they get by (coping mechanisms), what the interviewer said, and the system's summary of their vulnerability.
- Key vulnerability factors (implicit weighting): although no exact points were used, more weight was given to factors that indicate greater vulnerability: serious health problems or disability (especially if they prevent work or there is no support); caring for vulnerable dependents (infants, sick children); being the sole carer; living in a very unstable place (street, night-only shelter); being unable to cover essential needs (food, water, hygiene); using very risky ways to survive (such as begging); lacking valid documents; special risks (being LGBTIQ+, although the risk varies).
- Comparison: each case was compared with the others on which vulnerability factors it had, how severe they were, and how they combined.
- Verification: the information was checked for consistency and compared with the interviewer's opinion and the system's summary.
- Recommendations: who to include or exclude was decided on how vulnerable each case was compared with the others, and the rule of selecting only one or two.
Process-Guarantees.
- Transparency: clear information from standard forms is used; the factors and the reasoning are explained; objective information is distinguished from the interviewer's opinion.
- Accountability: official documents are used and this analysis creates a record; the tool is standardized for consistency; "A person reviews and takes the final decision, not the AI".
- Responsibility: the AI only helps analyze the information in the forms according to the rules given; the responsible person or committee decides in the end, applying the programme's criteria; the analysis uses no external information.
Neither panel contained anything about the household under review. Process-of-Thought described the language model's reading of the case, not the scoring engine that produced the score and recommendation. The two panels were always shown together, so their separate effects are unknown.
The review screen and the seven items answered after each assessment
Order on screen: registration record; Reasoning (a narrative of about 400 words in seven fixed parts: summary; possible contradictions or biases in the record; possible inconsistencies; reasons for inclusion; reasons for exclusion; relative vulnerability rating; final recommendation); statements (statements arm only); Answer (score, category, recommendation); then the items below. The Scorecard's determination was withheld until after submission, when a comparison screen presented it and the caseworker accepted or overrode it in turn.
| Item | Wording | Response |
|---|---|---|
EC1 | Rate the effectiveness of Cashy-AI for identifying and selecting the "right" individuals for the assistance | 1 to 5 |
EC2 | Do you agree or disagree with Cashy-AI's final recommendation for this case? | Agree / Disagree |
EC3 | Why do you disagree? Select all that apply (if Disagree) | Inaccurate or incorrect information; Irrelevant or incomplete information; Not appropriate or unexpected result; Unfair or unjust decision; Disrespects staff autonomy to change the decision; Lack of transparency or explainability; Unclear roles between staff and tool |
EC4 | Do you want to override Cashy-AI's final recommendation? (if Disagree) | Yes / No |
EC5 | Does Cashy-AI provide a correct and accurate recommendation? (correctness) | Completely inaccurate (1) to Completely accurate (5) |
EC6 | Does Cashy-AI provide content that is relevant to help you make an informed decision? (relevance) | Strongly disagree (1) to Strongly agree (5) |
EC7 | Does Cashy-AI provide a recommendation that is appropriate to the vulnerability profile of the case? (appropriateness) | Very inappropriate (1) to Very appropriate (5) |
Over-reliance is defined by EC4. The reasoning-answer gap is EC6 minus EC5 on the same assessment; a positive gap means the reasoning was rated above the answer.
Post-session principles questionnaire (five UN AI principles)
One item per principle, five-point labelled scale scored 1 to 5, followed by a required open comment. Forms were matched to assessments through the participant's self-selected name.
| Principle | Item | Labels 1 to 5 |
|---|---|---|
| Do no harm | Where does Cashy-AI fall on the following harm scale? | Actively harmful; Lacks safeguards; Basic safety; Minimizes harm; Proactively protective |
| Fairness and non-discrimination | Based on your analysis, where does Cashy-AI fall on the following fairness scale? | Discriminatory; Potentially biased; Mixed fairness; Largely fair; Highly fair |
| Human autonomy and oversight | Evaluate Cashy-AI's impact on human autonomy using the scale below. | Undermines staff autonomy; Limits staff autonomy; Basic staff autonomy; Respects staff autonomy; Enhances staff autonomy |
| Transparency and explainability | How transparent and explainable is Cashy-AI? | Opaque; Limited transparency; Partial transparency; Largely transparent; Fully transparent |
| Responsibility and accountability | Based on the oversight and the clarity of defined human responsibilities, where does Cashy-AI fall on the following accountability scale? | Unaccountable; Weak accountability; Basic accountability; Good accountability; Strong accountability |
Annex III: open research questions
- Which on-screen cues change what a caseworker does with a flawed recommendation, and which change only what they say about it? Can the Process-of-Thought and Process-Guarantees effects be separated?
- Is the reasoning-answer rating gap a usable individual-level flag for at-risk assessments, or only a group-level signal? What would make it sensitive enough to log in production?
- What is the smallest override audit (cases and caseworkers) that detects a 15-point fall in correct override at 80% power, and how should cases be chosen so that error direction is not confounded with case?
- How should override be recorded and monitored in production so that a falling rate is visible to a named owner, without turning override itself into a performance target?
- Can a model built on the synthetic sample expose, per case, which parts of the recommendation the features support (score, category) and which they do not (eligibility), and does showing that distinction change reliance?
- How can principle ratings, model cards and checklists be combined with behavioral evidence so that a self-assessment cannot rate a system well while its users override it less?
- What does an evaluation of the workflow look like from the side of the household that never sees the decision, and how could displaced people participate in setting the override standard?
Requirements
Ideally your design should meet these requirements. Annex IV of the brief lists the ethical and human-rights considerations that apply to assistance-targeting data and to AI-assisted decisions over vulnerable populations.
- Synthetic data only. Work exclusively on the synthetic sample and the released materials; do not attempt to reconstruct, link or re-identify households, caseworkers or offices. Present results as properties of the synthetic sample.
- Reasoning and answer kept apart. Any prototype or screen must present the system's reasoning and its recommendation as separate outputs and record the caseworker's judgment of each. "I agree" must not be accepted as a single act.
- Reliance decomposed. Report correct override, over-reliance, correct acceptance and under-reliance separately, on concordant and discordant cases, with Wilson or equivalent confidence intervals. Never a single agreement or accuracy rate alone.
- Reference standard named. "Correct" means agreement with the operation's own recorded determination, the standard caseworkers are accountable to. State that it is not ground truth about the household's need.
- Clustering and power. Assessments from the same caseworker are not independent; analyze at the participant level or with participant-clustered models, and report the minimum detectable effect rather than post hoc power.
- Modeling approach. If you build a model, document it (a model card is the expected format), report inclusion and exclusion recall separately, and say which decisions the features can and cannot support.
Deliverables
A repository
Code and a README that another operation could run.
A five-minute demo
Show the workflow, the measurement and what the institution would see.
A two-page note
The contrast you propose to test, the outcome measure, the sample needed, and what a positive and a negative result would mean for the operation.
Three points apply to every submission
The households whose data shaped the corpus never see the decisions made about them and cannot contest them, so any design must say where a wrong recommendation would be caught. The caseworkers are a small professional population and must not be identifiable from any analysis. A reference determination is the institution's standard, not the truth about a household's need.
Feasibility in the operation's stack
KoboToolbox forms, Python, and Power BI as the reporting standard. Solutions are judged on cost and maintainability by an information-management team, and on reproducibility: code and instructions that another operation could run.
How projects are judged
Four criteria. A technically impressive solution that fails to address the humanitarian problem will not perform well; a simple but practical, human-centred and feasible solution may score highly.
Challenge
Understanding of the humanitarian challenge and the role of human oversight.
Novelty
Originality and innovation.
Community impact
Consideration of affected populations, users and responsible AI principles.
Solution
Feasibility, scalability, usability and implementation potential.
What else the challenge poser weighs
The challenge poser also weighs fidelity to the research question; rigor of measurement (reliance decomposed rather than a single agreement rate, participants treated as the unit of randomization, confidence intervals reported); explainability and separation of reasoning from answer; feasibility in the operation's stack; do-no-harm (no real household data, no assistance at stake, no identification of caseworkers); cost and maintainability; and reproducibility.
Event
Organized by UNHCR Innovation and the University of Trento. Around 50 participants are expected, with mentors from the organizers, technical experts and UNHCR staff, and dedicated Q&A sessions with the UNHCR team.
Final submission requirements will be shared during the event. Teams should expect to submit a prototype, concept or framework; a short presentation or pitch; documentation explaining the solution; and a brief explanation of how impact would be measured.
FAQ
Do I need to be a programmer or data scientist?
No. The hackathon is intentionally multidisciplinary. Computer science, data science, sociology, international relations, law, psychology, design, communications and the humanities can all contribute meaningfully. Evaluation is not based solely on technical sophistication.
Is this an AI hackathon?
Partially. Teams may use AI, machine learning, analytics, design methods, governance approaches, monitoring tools, experimentation frameworks or other methods. The objective is not simply to build AI, but to solve the humanitarian problem presented in the challenge.
Do we need to improve the AI model?
Not necessarily. Improving model performance may be part of a solution, but it is not the primary objective. Strong solutions focus on human oversight, decision support, user experience, monitoring mechanisms, governance, accountability and responsible AI design rather than only increasing predictive accuracy.
Can we propose a governance or process solution instead of software?
Yes. Solutions may include digital prototypes, dashboards, evaluation frameworks, governance mechanisms, oversight processes, human-centred design approaches and testing methodologies. A fully implemented software product is not required.
What does a strong submission look like?
It understands the humanitarian problem, protects meaningful human oversight, demonstrates responsible AI principles, is feasible to implement, can be maintained realistically, and includes a clear method to measure effectiveness. Strong submissions focus on people, processes and outcomes rather than technology alone.
Does the solution need to be production-ready?
No. The goal is to explore, test and demonstrate ideas that could be developed further, not to deploy a complete enterprise solution within two days.
Is the data real?
No. The dataset contains synthetic records created from historical patterns. It does not contain real individuals and should not be interpreted as a representation of actual operational distributions.
Can we use external datasets?
Yes, provided their use is clearly documented and they do not introduce privacy, security, licensing or ethical concerns. Explain how external data contributes to solving the challenge.
Can we use generative AI?
Yes, provided you acknowledge its use, understand its limitations, remain accountable for outputs, and respect ethical and responsible AI principles. Using AI tools gives no automatic advantage in evaluation.
Why is human oversight important?
Humanitarian decisions can directly affect individuals' access to assistance and services. Technology may support decision-making, but responsibility remains with people. Effective human oversight helps reduce risks, identify errors and promote accountability. Human-in-the-loop means a person remains involved in reviewing, interpreting and taking responsibility for decisions supported by technology.
What humanitarian principles should guide our work?
Protection, Accountability to Affected Populations, inclusion, fairness, transparency, human oversight, data responsibility and do no harm. Do no harm means avoiding new risks or worsening existing ones: consider unintended consequences, exclusion risks and potential misuse of your solution.
Will technical projects automatically score higher?
No. A technically impressive solution that fails to address the humanitarian problem will not perform well. A simple but practical, human-centred and feasible solution may score highly. Social, legal, design and humanitarian insights count alongside technical expertise.
Reading list
Recommended before working with assistance-targeting data and designing AI-assisted decisions over vulnerable populations.
Humanitarian targeting and institutional AI principles
- UNHCR AI Approach (2025)
- UN-CEB. Principles for the Ethical Use of AI in the UN System (2022)
- UNHCR and WFP. Joint Principles on Targeting Assistance (2020)
- UNHCR. Mapping of Targeting and Prioritization Approaches (2023)
- ICRC and EPFL. From Principles to Practice: AI in Humanitarian Organizations (2026)
- Aiken et al. Machine learning and phone data can improve targeting of humanitarian aid. Nature (2022)
- Coppi, Moreno Jimenez and Kyriazi. Explicability of humanitarian AI. JIHA (2021)
- Scantamburlo, Baumann and Heitz. On prediction-modelers and decision-makers. AI & Society (2024)
Appropriate reliance, over-reliance and oversight
- Eckhardt et al. A Survey of AI Reliance. ACM Computing Surveys (2026)
- Passi, Dhanorkar and Vorvoreanu. Addressing Overreliance on AI (2025)
- Guo et al. A Decision Theoretic Framework for Measuring AI Reliance. FAccT (2024)
- Schemmer et al. Appropriate Reliance on AI Advice. IUI (2023)
- Lu and Yin. Human Reliance on ML Models When Performance Feedback is Limited. CHI (2021)
- Buçinca, Malaya and Gajos. To Trust or to Think. CSCW (2021)
- Vasconcelos et al. Explanations Can Reduce Overreliance on AI Systems. CSCW (2023)
- Fok and Weld. In search of verifiability. AI Magazine (2024)
- Eiband et al. The Impact of Placebic Explanations on Trust. CHI EA (2019)
- Gaube et al. Do as AI say. npj Digital Medicine (2021)
- Green. The flaws of policies requiring human oversight of government algorithms (2022)
- Mittelstadt. Principles alone cannot guarantee ethical AI. Nature MI (2019)
- Raji et al. Closing the AI accountability gap. FAT* (2020)
- Mitchell et al. Model Cards for Model Reporting. FAT* (2019)
Ethical and human-rights considerations
- Generative AI for Humanitarians (2023)
- Pizzi, Romanoff and Engelhardt. AI for humanitarian action: Human rights and ethics. IRRC (2020)
- Beduschi. Harnessing the potential of AI for humanitarian action. IRRC (2022)
- NLP for humanitarian action. Frontiers in Big Data (2023)
- Mikalef et al. Thinking responsibly about responsible AI. EJIS (2022)
- Moore et al. Concerning the Responsible Use of AI in the U.S. Criminal Justice System. CACM (2025)
Challenge poser and stakeholders
UNHCR's Innovation Service creates an enabling environment for innovation to flourish in UNHCR by equipping staff with the knowledge, resources and skills to draw on structured innovation to solve the most pressing challenges, with safe spaces for experimentation in field operations and at Headquarters.
Institutional stakeholders and frameworks
| Stakeholder | Role | Link |
|---|---|---|
| UNHCR | Institutional AI approach: transparent use of AI, always with human oversight | unhcr.org/digitalstrategy/ai-approach |
| UNHCR and WFP | Joint Principles on Targeting Assistance to Meet Food and Other Basic Needs | Joint Principles |
| UNHCR | Mapping of Targeting and Prioritization Approaches (how cash targeting is done) | Handbook |
| UN System Chief Executives Board | Principles for the Ethical Use of AI in the UN System (the five principles rated in the study) | unsceb.org |
| ICRC, EPFL, ETH Zurich | From Principles to Practice: AI in Humanitarian Organizations (AI Alignment Guide, 2026) | essentialtech.ch/reports |