{"submission_id":"245f4bd0-5d3c-41f4-9636-f3e0e9b5a8e8","submission_revision_id":"10f0c6b7-cb08-4511-8d05-0c9896b0b136","revision_version":2,"submission_status":"under_review","submitted_at":"2026-08-06T22:48:13.197764Z","submitter_name":"Yang Liu","submitter_email":"yliu8@nccu.edu","task":{"status":"submitted","version":2,"extra_fields":{"difficulty_self_test":{"model":"Claude Opus 4.8","harness":"Claude Code","estimated_score":"40–60%"}},"submitted_at":"2026-08-06T22:48:13.197764+00:00","industry_domain":"Machine Learning, Data Mining","task_short_name":"Pima_Indians_ML","operating_system":"Windows, Linux, MacOS","task_description":"# Project 2 Task Description\n\n## Objective\n\nBuild a reproducible machine-learning workflow for profiling and classifying diabetes outcomes using the supplied `pima-indians-diabetes.csv` dataset.\n\nThe workflow must identify invalid values, create complete-case and mean-imputed datasets, evaluate outliers, rank predictors with a Chi-square test, and compare six support vector machine configurations under a fully specified cross-validation protocol.\n\n## Software & Version\n\nUse the following environment, or a functionally equivalent environment that reproduces the required results:\n\n- Anaconda Python 3.12.7\n- Jupyter Notebook 7.2.x\n- pandas 2.2.x\n- NumPy 1.26.x\n- Matplotlib 3.9.x\n- scikit-learn 1.5.x\n\n## Dataset\n\nThe input file is `pima-indians-diabetes.csv`.\n\nThe CSV has no header row and contains 768 records, eight predictors, and one binary target. Assign these column names in order:\n\n| Position | Column | Description |\n|---:|---|---|\n| 0 | `preg` | Number of pregnancies |\n| 1 | `plas` | Plasma glucose |\n| 2 | `pres` | Diastolic blood pressure |\n| 3 | `skin` | Triceps skin-fold thickness |\n| 4 | `test` | Serum insulin |\n| 5 | `mass` | Body mass index |\n| 6 | `pedi` | Diabetes pedigree function |\n| 7 | `age` | Age |\n| 8 | `class` | Binary diabetes outcome |\n\nTreat the following values as invalid or missing:\n\n- `plas == 0`\n- `plas == 99999`\n- `pres == 0`\n- `skin == 0`\n- `test == 0`\n- `mass == 0`\n\nZeros in `preg`, `pedi`, `age`, and `class` are valid and must not be treated as missing.\n\n## Required Work\n\n### 1. Dataset Validation and Profiling\n\nLoad the dataset and report:\n\n- Shape\n- First records\n- Descriptive statistics\n- Predictor names\n- Target classes and class counts\n- Every location containing `99999`\n- Invalid or missing count for each affected predictor\n\nPreserve original row indices throughout the data-quality analysis.\n\n### 2. Exploratory Visualization\n\nCreate the following visual summaries of the original data:\n\n- Boxplots\n- Histograms\n- Density plots\n- Scatterplot matrix\n\nAlso create density plots for the complete-case dataset.\n\nPlots must use readable labels and identify the represented variables. Exact colors and plot dimensions are not prescribed.\n\n### 3. Missing-Data Treatments\n\nCreate two analysis datasets after converting the specified invalid values to `NaN`.\n\n#### Complete-case dataset\n\nRemove every row containing at least one missing predictor value.\n\nReport:\n\n- Retained shape\n- Retained class distribution\n\n#### Mean-imputed dataset\n\nReplace each missing predictor value with the arithmetic mean of the corresponding nonmissing predictor values.\n\nReport:\n\n- Imputation mean for each affected predictor\n- Final shape\n- Confirmation that no missing values remain\n- Confirmation that the `class` column is unchanged\n\n### 4. Outlier Assessment\n\nUsing the complete-case dataset, identify observations more than three population standard deviations from the corresponding column mean.\n\nUse population standard deviation, equivalent to `np.std(..., ddof=0)`.\n\nReport:\n\n- Outlier count for every column\n- Columns containing at least one outlier\n- Original source-row indices containing one or more outlying values\n- Total number of column-level outlier occurrences\n\nDo not remove or replace these outliers for the required model comparison.\n\n### 5. Chi-Square Feature Analysis\n\nRun Chi-square univariate feature analysis on the eight predictors in the mean-imputed dataset.\n\nBecause all resulting values are nonnegative, no additional scaling is required for the Chi-square calculation.\n\nReport:\n\n- Chi-square score and p-value for all eight predictors\n- Ranking from highest to lowest score\n- Six highest-scoring predictors\n- Two unselected predictors\n\nFeature selection is performed once on the complete mean-imputed dataset. The resulting six fixed predictors are used in the six-feature SVM configuration. This configuration is therefore directly based on the Chi-square result; it is not a manual `plas`/`pedi` removal experiment.\n\n### 6. SVM Evaluation Protocol\n\nUse:\n\n- `SVC(kernel=\"rbf\", C=1.0, gamma=\"scale\")`\n- `KFold(n_splits=10, shuffle=True, random_state=42)`\n- Accuracy scoring\n- `n_jobs=1`\n- Population standard deviation with `ddof=0`\n\nFor all configurations containing 768 records, reuse the same deterministic KFold split definition. The complete-case configuration uses the same KFold settings but necessarily has different fold membership because it contains fewer records.\n\nCompare exactly these six configurations:\n\n1. **Raw Dataset**\n   - Original eight predictors\n   - Invalid zeros and `99999` retained as numeric values\n   - No scaling\n\n2. **Complete-Case Dataset**\n   - Rows with invalid predictor values removed\n   - All eight predictors\n   - No scaling\n\n3. **Chi-Square Six-Feature Dataset**\n   - Mean-imputed dataset\n   - Exactly the six highest-scoring Chi-square predictors\n   - No scaling\n\n4. **Mean-Imputed Dataset**\n   - All eight mean-imputed predictors\n   - No scaling\n\n5. **Normalized Mean-Imputed Dataset**\n   - All eight mean-imputed predictors\n   - `MinMaxScaler()` applied to all eight predictors\n   - Scaler fitted only on each training fold by using a scikit-learn `Pipeline`\n\n6. **Standardized Mean-Imputed Dataset**\n   - All eight mean-imputed predictors\n   - `StandardScaler()` applied to all eight predictors\n   - Scaler fitted only on each training fold by using a scikit-learn `Pipeline`\n\nDo not use row-wise `Normalizer`, add feature arrays together, scale only selected columns, reorder the target, or otherwise reproduce the flawed normalization and standardization logic from the original reference notebook.\n\n### 7. Performance Reporting\n\nFor every configuration, report:\n\n- Ten fold-level accuracy scores\n- Mean accuracy\n- Population standard deviation\n- Change in mean accuracy relative to the Raw Dataset\n\nDisplay summary values to four decimal places and explain:\n\n- Which configuration has the highest mean accuracy\n- Whether complete-case deletion improves performance\n- Whether the fixed Chi-square feature subset changes performance\n- Whether min-max normalization improves performance\n- Whether standardization improves performance\n\nAvoid causal or clinical claims.\n\n## Required Deliverables\n\nSubmit a Jupyter Notebook that:\n\n- Runs from beginning to end without errors\n- Requires no internet access or manual input\n- Uses no machine-specific absolute path\n- Contains the complete data profile\n- Contains all required plots\n- Produces complete-case and mean-imputed datasets\n- Reports the outlier analysis\n- Reports all Chi-square scores and selected features\n- Evaluates all six SVM configurations\n- Produces a six-row performance comparison table\n- Includes concise interpretation\n\n## Expected Completion Time\n\n- Expected range: 3–5 hours\n- Recommended estimate: **4 hours**","software_licensing":"Free/Open Source","evaluation_criteria":null,"verification_method":"# Project 2 Verification Method\n\nThe submission passes only when the notebook executes successfully and matches the following protocol and expected results.\n\n## 1. Canonical Input\n\n- Filename: `pima-indians-diabetes.csv`\n- Shape: `(768, 9)`\n- Predictors: `preg`, `plas`, `pres`, `skin`, `test`, `mass`, `pedi`, `age`\n- Target: `class`\n- Target values: `[0, 1]`\n- Class `0`: `500`\n- Class `1`: `268`\n\nCanonical SHA-256:\n\n```text\nc87618e1ffa05e7891132baaf926ed541f4c96d2753e54064c9a6228327a241e\n```\n\nThe first records and descriptive statistics must match the canonical CSV. Numeric descriptive statistics are accepted within an absolute tolerance of `1e-6`.\n\n## 2. Invalid and Missing Values\n\n### Exact `99999` locations\n\n| Column | Original row index |\n|---|---:|\n| `plas` | 500 |\n| `plas` | 549 |\n\nNo other `99999` values are present.\n\n### Exact invalid-value counts\n\n| Predictor | Zero count | `99999` count | Total invalid/missing |\n|---|---:|---:|---:|\n| `plas` | 5 | 2 | 7 |\n| `pres` | 35 | 0 | 35 |\n| `skin` | 227 | 0 | 227 |\n| `test` | 374 | 0 | 374 |\n| `mass` | 11 | 0 | 11 |\n\nThese values must match exactly.\n\n## 3. Missing-Data Treatments\n\n### Complete-case dataset\n\nExpected results:\n\n- Shape: `(391, 9)`\n- Class `0`: `261`\n- Class `1`: `130`\n\n### Mean-imputation values\n\n| Predictor | Expected mean |\n|---|---:|\n| `plas` | 121.604467805519 |\n| `pres` | 72.405184174625 |\n| `skin` | 29.153419593346 |\n| `test` | 155.548223350254 |\n| `mass` | 33.308850726552 |\n\nEach mean must match within `1e-6`.\n\nThe mean-imputed dataset must:\n\n- Have shape `(768, 9)`\n- Contain no missing values\n- Preserve the original `class` column exactly\n\n## 4. Outlier Results\n\nUse the complete-case dataset and the three-population-standard-deviation rule.\n\n| Column | Expected outlier count |\n|---|---:|\n| `preg` | 1 |\n| `plas` | 0 |\n| `pres` | 5 |\n| `skin` | 1 |\n| `test` | 8 |\n| `mass` | 1 |\n| `pedi` | 1 |\n| `age` | 2 |\n| `class` | 0 |\n\nAdditional exact results:\n\n- Total column-level outlier occurrences: `19`\n- Columns containing outliers: `preg`, `pres`, `skin`, `test`, `mass`, `pedi`, `age`\n- Unique original source-row indices:\n\n```text\n[8, 13, 18, 43, 125, 177, 228, 247, 286, 409,\n 445, 459, 499, 584, 597, 655]\n```\n\nRow `499` contains four outlying variables but must appear only once in the unique row-index list.\n\n## 5. Corrected Chi-Square Results\n\nThe Chi-square analysis must use the mean-imputed eight-predictor dataset rather than the raw sentinel-coded matrix.\n\n| Rank | Predictor | Chi-square score | p-value |\n|---:|---|---:|---:|\n| 1 | `test` | 1638.618149516463 | 0.000000000000e+00 |\n| 2 | `plas` | 1431.012534816713 | 0.000000000000e+00 |\n| 3 | `age` | 173.567621931765 | 1.230292500790e-39 |\n| 4 | `skin` | 94.245703109462 | 2.786910696295e-22 |\n| 5 | `preg` | 89.502357715718 | 3.062732522753e-21 |\n| 6 | `mass` | 53.584146305497 | 2.477506347804e-13 |\n| 7 | `pres` | 42.749956325223 | 6.220327351955e-11 |\n| 8 | `pedi` | 0.164009894558 | 6.854912192122e-01 |\n\nExpected six selected predictors, in score order:\n\n```text\n['test', 'plas', 'age', 'skin', 'preg', 'mass']\n```\n\nExpected unselected predictors:\n\n```text\n['pres', 'pedi']\n```\n\nScores and p-values must match within `1e-6`.\n\n## 6. Required SVM Implementation\n\nEvery SVM must use:\n\n```text\nSVC(kernel=\"rbf\", C=1.0, gamma=\"scale\")\n```\n\nCross-validation must use:\n\n```text\nKFold(n_splits=10, shuffle=True, random_state=42)\n```\n\nAdditional requirements:\n\n- Accuracy scoring\n- `n_jobs=1`\n- Population standard deviation calculated with `np.std(scores, ddof=0)`\n- `MinMaxScaler` and `StandardScaler` applied to all eight mean-imputed predictors\n- Scaling performed inside a scikit-learn `Pipeline`\n- No row-wise `Normalizer`\n- No feature-array addition\n- No partial-column scaling\n- No target reordering\n\nThe Chi-square six-feature configuration must use the six predictors listed in Section 5. It must not manually remove `plas` and `pedi`.\n\n## 7. Corrected Fold-Level Accuracy Scores\n\n### Raw Dataset\n\n```text\n[0.649350649351, 0.636363636364, 0.675324675325,\n 0.714285714286, 0.636363636364, 0.571428571429,\n 0.649350649351, 0.740259740260, 0.592105263158,\n 0.644736842105]\n```\n\n### Complete-Case Dataset\n\n```text\n[0.875000000000, 0.615384615385, 0.743589743590,\n 0.794871794872, 0.692307692308, 0.717948717949,\n 0.717948717949, 0.923076923077, 0.743589743590,\n 0.871794871795]\n```\n\n### Chi-Square Six-Feature Dataset\n\n```text\n[0.727272727273, 0.779220779221, 0.675324675325,\n 0.831168831169, 0.805194805195, 0.649350649351,\n 0.779220779221, 0.753246753247, 0.710526315789,\n 0.776315789474]\n```\n\n### Mean-Imputed Dataset\n\n```text\n[0.727272727273, 0.766233766234, 0.688311688312,\n 0.831168831169, 0.818181818182, 0.649350649351,\n 0.766233766234, 0.740259740260, 0.697368421053,\n 0.776315789474]\n```\n\n### Normalized Mean-Imputed Dataset\n\n```text\n[0.701298701299, 0.818181818182, 0.727272727273,\n 0.831168831169, 0.779220779221, 0.636363636364,\n 0.805194805195, 0.766233766234, 0.684210526316,\n 0.776315789474]\n```\n\n### Standardized Mean-Imputed Dataset\n\n```text\n[0.688311688312, 0.805194805195, 0.714285714286,\n 0.831168831169, 0.766233766234, 0.688311688312,\n 0.779220779221, 0.727272727273, 0.671052631579,\n 0.776315789474]\n```\n\nEach fold-level score must match within `1e-6`.\n\n## 8. Corrected Performance Summary\n\n| Configuration | Mean accuracy | Population SD | Change from raw |\n|---|---:|---:|---:|\n| Raw Dataset | 0.650956937799 | 0.047788461542 | 0.000000000000 |\n| Complete-Case Dataset | 0.769551282051 | 0.090659814721 | 0.118594344252 |\n| Chi-Square Six-Feature Dataset | 0.748684210526 | 0.054598259391 | 0.097727272727 |\n| Mean-Imputed Dataset | 0.746069719754 | 0.054423785091 | 0.095112781955 |\n| Normalized Mean-Imputed Dataset | 0.752546138072 | 0.060131947501 | 0.101589200273 |\n| Standardized Mean-Imputed Dataset | 0.744736842105 | 0.051771970169 | 0.093779904306 |\n\nTolerances:\n\n- Fold score: `1e-6`\n- Mean accuracy: `0.001`\n- Population SD: `0.001`\n- Change from raw: `0.001`\n\nThe displayed summary must round metrics to four decimal places.\n\n## 9. Interpretation Checks\n\nThe written interpretation must correctly state that:\n\n- The Complete-Case Dataset has the highest mean accuracy: `0.7696`.\n- The Normalized Mean-Imputed Dataset is the strongest full-sample configuration: `0.7525`.\n- The Chi-Square Six-Feature Dataset reaches `0.7487`.\n- Mean imputation without scaling reaches `0.7461`.\n- Standardization reaches `0.7447`.\n- All corrected non-raw configurations outperform the Raw Dataset mean of `0.6510`.\n- Complete-case deletion changes the sample from 768 to 391 records, so its higher accuracy must not be interpreted as evidence that deletion is universally preferable.\n- No causal or clinical conclusion may be drawn from these predictive comparisons alone.\n\n## 10. Visualization Checks\n\nThe notebook must contain:\n\n- A boxplot layout for the original variables\n- A histogram layout\n- A density-plot layout for the original variables\n- A scatterplot matrix\n- A density-plot layout for the complete-case data\n\nThe grader must verify content and labels but must not require a specific color palette or exact figure dimensions.\n\n## 11. Execution and Completeness\n\nThe notebook must:\n\n- Run all cells without errors\n- Require no internet access or manual editing\n- Produce the complete-case dataset with 391 records\n- Produce the mean-imputed dataset with 768 records\n- Report all corrected Chi-square results\n- Select exactly six predictors\n- Evaluate exactly six SVM configurations\n- Produce ten fold scores for every configuration\n- Produce a six-row performance table\n- Use leakage-safe scaling pipelines\n- Produce the required visualizations\n\nThe task passes only when all required outputs and numerical checks succeed.\n\n","software_and_version":"Anaconda Python 3.12.7 with Jupyter Notebook 7.2.x, pandas 2.2.x, NumPy 1.26.x, Matplotlib 3.9.x, and scikit-learn 1.5.x.","specific_requirements":null,"expected_completion_time":null},"files":[{"file_id":"842aa28b-fce2-49e6-9cd5-bf39c0b0a357","filename":"pima-indians-diabetes.csv","category":"input_materials","content_type":"text/csv","size_bytes":23777},{"file_id":"956f73b8-75f1-49af-8577-547428f200d0","filename":"Pima.ipynb","category":"reference_output","content_type":"application/octet-stream","size_bytes":548353}],"deep_review":{"overall_verdict":"Accept","review_text":"**Overall Verdict:** Accept — The task is executable and objectively verifiable from an intact canonical dataset and detailed corrected rubric, although the uploaded notebook is a superseded reference.\n\n**Completeness Check:**\n- Instruction: Complete — The workflow, six model configurations, preprocessing rules, cross-validation protocol, plots, reporting requirements, and final deliverable are fully specified.\n- Inputs: Complete — The CSV is a readable 23,777-byte numeric file with 768 nine-field records; its SHA-256, schema, class counts, invalid-value counts, and sentinel locations match the verification contract.\n- Toolchain: Complete — The specified Python/Jupyter stack is free, cross-platform, available offline, and requires no specialized hardware or services.\n- Reference output: Unclear — `Pima.ipynb` is valid notebook JSON with 20 code cells, 19 stored outputs, and five intact labeled plots, but it contains the explicitly superseded raw Chi-square, manual feature removal, row-wise normalization, partial scaling, and target-reordering logic.\n- Rubric: Complete — It provides exact expected values, tolerances, fold scores, implementation constraints, visualization checks, execution requirements, and interpretation criteria.\n\n**File and Task Reachability:**\nThe attachment classification is `partially_reachable`: both `pima-indians-diabetes.csv` and `Pima.ipynb` were fully inspected and are readable, unencrypted, and free of unsafe active content, but the notebook is not a canonical reference for the revised protocol. No attachment or embedded plot content was left unreviewed. The task classification is `reachable`: a contributor can use the CSV with the specified open-source stack to produce the notebook, and evaluation can follow the exact data-quality, Chi-square, cross-validation, tabular, and visualization checks. The imputation means, complete-case results, outlier results, and Chi-square scores were independently confirmed. A compatible Python runtime was unavailable in the review environment, so the six SVM fold sequences were not independently regenerated; this is a reviewer-tooling limitation rather than a submission defect. License keys, accounts, tokens, and other credentials are not required from the contributor. Confidential, proprietary, or internal material may be evaluated as submitted evidence without requiring additional disclosure.\n\n**Special Screening:**\nNone — public-benchmark status could not be independently verified within the permitted evidence boundary.\n\n**Recommended Changes:**\nThe canonical input and corrected verification contract are strong, and no material changes are required. As a maintenance improvement, replace `Pima.ipynb` with a corrected reference notebook or label it clearly as a legacy, noncanonical example so its stored outputs cannot be mistaken for the required results.","reviewed_at":"2026-08-15T00:23:46.314009Z"},"claim":null}