Skip to content

StopTB camp laptop data sync – = sync, inventory upload fixes and master download - #201

Merged
vanitha1822 merged 9 commits into
release-3.12.0from
vb/3.12.0-auto-sync
Oct 21, 2026
Merged

vanitha1822 merged 9 commits into
release-3.12.0from
vb/3.12.0-auto-sync

Conversation

@vishwab1

Copy link
Copy Markdown
Member

📋 Description

JIRA ID: AMM-2491

Please provide a summary of the change and the motivation behind it. Include relevant context and details.


✅ Type of Change

  • 🐞 Bug fix (non-breaking change which resolves an issue)
  • ✨ New feature (non-breaking change which adds functionality)
  • 🔥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 🛠 Refactor (change that is neither a fix nor a new feature)
  • ⚙️ Config change (configuration file or build script updates)
  • 📚 Documentation (updates to docs or readme)
  • 🧪 Tests (adding new or updating existing tests)
  • 🎨 UI/UX (changes that affect the user interface)
  • 🚀 Performance (improves performance)
  • 🧹 Chore (miscellaneous changes that don't modify src or test files)

ℹ️ Additional Information

Please describe how the changes were tested, and include any relevant screenshots, logs, or other information that provides additional context.

vishwab1 and others added 9 commits October 15, 2026 09:38
…B laptops

- Upload (van to central) every 30 min and master download every 6 h,
  using the central connection of the automatic BenID download (no stored
  password; SyncedBy = operator who connected the laptop). Own thread and
  lock; skips while a manual master download runs.
- GET /dataSyncActivity/autoSyncStatus: last runs, last error, and rows
  central rejected (Processed='F') per table and SyncFailureReason.
- Active only where stoptb.enforce.vanid=true and stoptb.van.id>0; manual
  Upload / Download Masters unchanged. 6 unit tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refills and stock issued at central for this van's store (indents, indent
issues, stock transfers, stock entries) reach the laptop without pressing
Inventory Data Download. Runs after each automatic upload; its own failure
does not undo the upload; shown in autoSyncStatus. Switch:
sync.autoInventoryDownload.enabled (default true). 2 new unit tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…k transfers

The loop over the 5 tables also did i++ at the end of each pass, so only
t_indent, t_indentissue and t_itemstockentry were downloaded;
t_indentorder and t_stocktransfer never reached the van.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pload

m_synctabledetail maps van columns to server columns by position (the van's
own ID sits where the server list has VanSerialNo); since the by-name record
mapping, VanSerialNo was stored NULL, so a re-sent row was inserted again
(duplicates) and joins on VanSerialNo broke. VanSerialNo now takes the van
row ID and SyncFacilityID the row's store when not sent by name
(dataSync.fillVanKeys, default true).

Master download on central logs the row count instead of every row (the
661k-row village table exhausted the heap).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…by default

- StopTB laptops (stoptb.enforce.vanid=true): service-line and van masters
  (hubs, vans, stores, items, item-store mapping, roles) are downloaded in full
  for the van's line, so a laptop set up from a copied database gets them all;
  shared masters (villages) stay incremental. Other MMU laptops unchanged.
- Automatic upload / master download / inventory download are off by default
  (manual sync until approved); when on, the automatic master download fetches
  only the inventory masters (sync.autoMasterDownload.inventoryOnly=true).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ion IDs

A van sends its own IDs in BenVisitID / PrescriptionID; on central those
numbers belong to other rows, so prescriptions and prescribed drugs were
linked to another van's visit/prescription, or rejected by the foreign key
when the number did not exist. With dataSync.translateVanLinks=true the link
is replaced by the central ID of the row the same van uploaded with that
VanSerialNo; if that row is not on central yet the record is not saved and is
sent again on the next upload. Off by default (other MMU servers unchanged);
to be switched on for the StopTB central.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ventory group

Inventory tables are configured in group 5 (old lists: stock-outs without
FacilityID/SyncFacilityID, patient issues without SyncFacilityID, batch insert
fails) and in group 9 Inventory (complete). Group 5 runs first and marks the
rows uploaded, so central stored stock-outs without their camp store and
per-store reports missed them. On StopTB laptops (stoptb.enforce.vanid=true) a
table that a later group also uploads is skipped. Other MMU laptops unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…column and continues past a failing table

The laptop sync list still names tb_diagnostic_order.reason_for_refusal; V125 renamed it to
reason_to_close and the whole upload stopped with bad SQL grammar. The column name the table
really has is used for the van and server column lists, and a table that cannot be read is
recorded as failed while the other tables still upload. Both only on a StopTB laptop
(stoptb.enforce.vanid).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 21, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ccd73874-554f-4f4f-bb45-a66e9eadcd9c
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)
C Security Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Comment on lines +158 to +160
List<Map<String, Object>> rows = jdbcTemplate.queryForList("SELECT '" + table.getTableName()
+ "' AS tableName, SyncFailureReason AS reason, COUNT(*) AS failedRows, DATE_FORMAT(MAX(SyncedDate), '%d-%m-%Y %H:%i') AS lastTried "
+ "FROM " + qualified + " WHERE Processed = 'F' GROUP BY SyncFailureReason");
Comment on lines 186 to 187
@vanitha1822
vanitha1822 merged commit bbe70ed into release-3.12.0 Oct 21, 2026
2 of 5 checks passed
drtechie pushed a commit that referenced this pull request Nov 2, 2026
* fix: add clinical staff information

* fix: pom version

* feat(stoptb): add Nikshay ID Generator CSV export/import

Adds a date-range-scoped export of Stop TB beneficiaries as a CSV
formatted for the Nikshay ID Generator app, and an import endpoint to
write the portal-generated Nikshay IDs back onto those beneficiaries.

Scoped by visit-date range only, no vanID/servicePointID — MMU runs
one local database per van, so everything in it already belongs to
the current van/service point.

Location columns (village/healthFacility/tu/district/state) are
resolved against Nikshay's own, isolated location hierarchy
(m_nikshay_village -> m_nikshay_facility -> m_nikshay_tu ->
m_nikshay_district -> m_nikshay_state), not AMRIT's standard masters
— a Stop TB beneficiary's I_bendemographics.DistrictBranchID actually
holds their Nikshay Village ID (the same overload Common-API's
NikshayAddressResolver relies on), so every other location field is
derived by walking up that one chain. A beneficiary whose village
doesn't resolve all the way to a state is silently left out of the
export, same as one who already has a Nikshay ID.

New endpoints:
- GET /stopTb/nikshay/exportBeneficiariesCsv?fromDate=&toDate=
- POST /stopTb/nikshay/importResultsCsv?visitDate=&file=

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(stoptb): store Nikshay ID on tb_suspected, match import rows by phone+name

Two corrections after checking real data:

1. nikshay_id now lives on tb_suspected, not tb_stoptb_diagnostics — both
   tables have the column, but tb_suspected is the one this feature is
   meant to use going forward. Both the export's already-has-an-ID check
   and the import's write path now target tb_suspected.benRegID.

2. A real Nikshay ID Generator results file has no beneficiary ID column
   of any kind — only the original template columns plus its own
   generatedId/status/durationSec/error. The import can no longer assume
   benRegId round-trips through untouched. Rows are now matched back to a
   beneficiary by normalized primaryPhone + case-insensitive firstName
   (NikshayExportRepository.findMatchingBeneficiaryIds); anything other
   than exactly one match is left for manual review rather than guessed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(stoptb): rewrite export query against verified real schema

Every table the export previously used for beneficiary/visit data turned
out not to exist on a real server (i_beneficiary, I_bendemographics,
tb_stoptb_visit) — confirmed via BadSqlGrammarException in production
logs and direct schema inspection. Rewritten against tables and columns
verified with real data:

- Visit filter: t_benvisitdetail (VisitCategory = 'Stop TB'), not
  tb_stoptb_visit — confirmed 511 real rows vs. zero.
- Beneficiary identity: MMU's datasource only connects to db_iemr, but
  beneficiary data lives in db_identity on the same physical server —
  reached via fully-qualified cross-schema table names:
  i_beneficiarymapping (BenRegId, unique) -> BenDetailsId ->
  i_beneficiarydetails (name/DOB/gender/caste/occupation/income/HIV,
  already denormalized, no extra master-table joins needed) and
  BenAddressId -> i_beneficiaryaddress (address/village/pincode), plus
  BenContactsId -> i_beneficiarycontacts (phone).
- New confirmed real gap: many van-registered beneficiaries
  (i_beneficiarymapping.CreatedBy = 'reglocal') have a mapping row but
  their detail/address rows never synced to db_identity, sometimes for
  days. These are now silently skipped (no FirstName resolved), same
  category as an unresolved location — renamed
  countUnresolvedLocation -> countNotReadyToExport to reflect both cases.
- Import's phone+name matching updated to the same verified join path.

Tested against real data on a live server: 15 of 20 today's-date
beneficiaries correctly skipped (identity never synced), 5 correctly
resolved end-to-end including the full Nikshay location chain, producing
valid CSV rows.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(stoptb): fix CSV streaming response (No converter for [...Lambda...])

exportBeneficiariesCsv was declared ResponseEntity<?> so it could return
either a StreamingResponseBody (success) or a plain String (bad request/
error). Spring's StreamingResponseBodyReturnValueHandler decides whether
to special-case the response based on the method's *declared* generic
return type, not the runtime object — a wildcard resolves to Object,
StreamingResponseBody isn't assignable from Object, so it falls through
to normal HttpMessageConverters, which can't serialize a raw lambda:
"No converter for [...NikshayExportController$$Lambda...] with preset
Content-Type 'text/csv'" (confirmed in production logs, 2026-08-18).

Fixed by keeping the return type consistently
ResponseEntity<StreamingResponseBody> and wrapping error messages as a
StreamingResponseBody that writes the message as bytes, instead of
returning a plain String on any path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(stoptb): allow REGISTRAR role on Nikshay export/import endpoints

@PreAuthorize only listed clinical roles (Nurse/Doctor/Pharmacist/Lab
Technician/etc.), copied from sn/nikshaya's original controller without
verifying against real Stop TB camp roles. Confirmed in production:
AccessDeniedException for a real Stop TB user (stoptb1) whose roles
(checked directly via v_userservicerolemapping) include Registrar,
Nurse, Doctor, Pharmacist, Lab Technician, Data Sync — Registrar was the
one missing from the allow-list. Downloading/uploading a beneficiary CSV
is fundamentally a registrar task, so this was very likely the actual
active session role denied by AccessDeniedException in the logs.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(stoptb): remove @PreAuthorize role gate on Nikshay export/import

Role-based gating proved fragile — wrong role list on the first attempt,
still AccessDeniedException after adding REGISTRAR. SecurityConfig's
anyRequest().authenticated() still requires a logged-in user; dropping
the role-specific check per explicit request rather than guessing a
third time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(stoptb): write CSV synchronously instead of via StreamingResponseBody

Root cause of the persistent AccessDeniedException (survived removing
@PreAuthorize entirely): StreamingResponseBody makes Spring process the
response body on a Servlet async re-dispatch. Confirmed from production
logs (2026-08-18) — two AccessDeniedExceptions per request, the second
via ApplicationDispatcher/AsyncContextImpl. Spring Security's own core
filter chain is registered for ASYNC dispatch by default and re-runs,
but this app's custom JwtUserIdValidationFilter/RoleAuthenticationFilter
(which read the JWT cookie and establish who's logged in) are not —
AnonymousAuthenticationFilter fires instead on the second pass, and the
global anyRequest().authenticated() rule denies it.

Fixed by writing the CSV directly and synchronously onto
HttpServletResponse instead of returning a StreamingResponseBody —
avoids async dispatch entirely, so there's no second security pass to
fail.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(stoptb): read phone/address from the columns actually populated

Verified against real data: PhoneNum1 and CurrAddressValue are empty on
every real beneficiary checked — the actual data lives in
PreferredPhoneNum (phone) and CurrAddrLine1/2/3 + CurrHabitation
(address), confirmed by direct query on real records. Falls back through
PhoneNum1/PhoneNum2 if PreferredPhoneNum is empty. Address line parts are
individually NULLIF'd before CONCAT_WS, since blank (not NULL) line
fields otherwise leave stray ", ," artifacts.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix(stoptb): import phone matching missed PreferredPhoneNum too

findMatchingBeneficiaryIds only checked c.PhoneNum1 — same bug already
fixed on the export side (commit 79d8422) but missed here. Confirmed
in production: two real beneficiaries (17877 'K', 17875 'U') with
PhoneNum1 IS NULL but PreferredPhoneNum populated ('9999999999') both
came back "no beneficiary found" on upload. Now checks
PreferredPhoneNum/PhoneNum1/PhoneNum2, same as the export's column
preference order. Verified against real data: 17877 now matches
correctly for phone '9999999999' + firstName 'K'.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat(stoptb): use benRegId directly when present, fall back to phone+name

If a results row carries a parseable benRegId (e.g. a manually-rebuilt
test file, or a future ID Generator version that keeps it), use it
directly — exact match, no ambiguity possible, skips the phone+name
lookup entirely. Falls back to the existing phone+name matching only
when benRegId is absent or unparseable, which is what a genuine Nikshay
portal results file looks like (confirmed no ID column at all).

Makes test files with benRegId (e.g. a rebuilt export) match instantly
and unambiguously, while real portal files keep working exactly as
before via the phone+name path.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: add the endpoints and schema for downsync from central to local

* fix: downsync implementation

* fix: build issue

* fix: lastmoddate updation

* fix: update the response of downsync

* fix: update the continuous call on check sync progress

* fix: column name issue for stoptb tables

* fix: issue in stoptb tables

* fix: add central id

* fix: FK Mapping issue

* fix: downsync issue

* fix: FK issue in i_ beneficiarymapping table

* fix: include U for downsync again

* fix: revert central id for m_beneficiaryregidmapping

* fix: conflict section

* fix: manage conflicts and add F flag for resync

* fix: pom version update

* fix: add the condition to block the download from different van

* fix: resolve foreign key violation

* fix: retry failed records

* fix: update 'F' flag

* fix: sonarqube comments

* fix: resolve sonarcube comments

* fix: issue in m_beneficiaryregidmapping table

* Adding api's for syncing diagnostic documents to s3

* fix: add the endpoints and schema for downsync from central to local

* fix: downsync implementation

* fix: build issue

* fix: lastmoddate updation

* fix: update the response of downsync

* fix: update the continuous call on check sync progress

* fix: column name issue for stoptb tables

* fix: issue in stoptb tables

* fix: add central id

* fix: FK Mapping issue

* fix: downsync issue

* fix: FK issue in i_ beneficiarymapping table

* fix: include U for downsync again

* fix: revert central id for m_beneficiaryregidmapping

* fix: conflict section

* fix: manage conflicts and add F flag for resync

* fix: pom version update

* fix: add the condition to block the download from different van

* fix: resolve foreign key violation

* fix: retry failed records

* fix: update 'F' flag

* fix: sonarqube comments

* fix: add clinical staff information

* fix: pom version

* fix: resolve sonarcube comments

* fix: issue in m_beneficiaryregidmapping table

* Adding api's for syncing diagnostic documents to s3

* Adding api's for syncing diagnostic documents to s3

* feat(stoptb): record whether AMRIT caused each Nikshay ID to be created

The results-CSV import already wrote the portal's Nikshay ID onto
tb_suspected, but nothing distinguished an ID the portal generated for us
from one that already existed there and was only matched back. Counting
"IDs created by this application" therefore meant inferring it from the
ID's presence, which over-counts every pre-existing ID.

Each written row now also sets tb_suspected.nikshay_created_by_amrit --
true for a "success" row, false for a "skipped" row (portal-detected
duplicate). Rows left for manual review or genuinely failed still write
nothing at all, so the existing matching, review and failure behaviour is
unchanged; "failed" maps to false for completeness only, as those rows
have no ID to write.

The import response gains createdByAmrit/alreadyOnNikshay counts, a
breakdown of the existing `updated` total that always sums to it, so the
figure is available without a query. Additive -- no existing field
changes meaning.

Requires AMRIT-DB V107 for the column. Note the flag is not yet in the
van<->central sync column mapping, so it stays van-local until that row
is updated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore(stoptb): remove added code comments

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(stoptb): store Nikshay ID on i_beneficiarydetails, not tb_suspected

The import previously fabricated a brand-new tb_suspected row for any
beneficiary who didn't already have one, just to hold the generated
Nikshay ID. tb_suspected is a clinical table (is_confirmed, is_referred,
reason_for_suspicion, etc.) — inserting a bare row there falsely implied
the beneficiary had been assessed as a TB suspect when no such
assessment ever happened, and every beneficiary needs a Nikshay ID
regardless of suspect status.

A Nikshay ID is a beneficiary-level identifier, not a clinical fact, so
it now lives on db_identity.i_beneficiarydetails (nikshayId,
nikshayCreatedByAmrit — new columns, migration to follow separately in
AMRIT-DB) alongside similar existing identifiers like rchid on
i_beneficiarydetails_rmnch.

- NikshayExportRepository: existingNikshayId now reads d.nikshayId
  (already-joined i_beneficiarydetails) instead of a tb_suspected
  subquery. Replaced findLatestSuspectedId/updateNikshayId/
  insertSuspectedWithNikshayId with a single writeNikshayId() — always
  an UPDATE via i_beneficiarymapping.BenRegId -> BenDetailsId, never an
  INSERT, since every beneficiary reaching this point already has a
  synced i_beneficiarydetails row.
- NikshayImportService: calls writeNikshayId() directly and reports a
  failed row if it updates zero rows, instead of silently dropping the
  generated ID.

tb_suspected is no longer touched by this feature.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* Dropping duplicate columns in table tb_stoptb_general_examination

* Merge 3.4.2 with 3.9.0 (#193)

* fix: Added proper space in the string

* Fix the issue in saving ANC Category Beneficiary (#146)

* fix: anc care issue

* fix: alignment

* fix: datasync issue

* fix: datasync issue

* fix: sync timeout issue

* Release 3.6.0 to Release-3.6.1 (#159)

* fix: amm-2147 anc care visit is not saving in the nurse section

* fix: amm-2147 anc care visit is not saving in the nurse section

* Revert "Sn/3.6.0"

* fix: amm-2147 anc care visit is not saving in the nurse section

* docs: update README and sync package version with pom (#169)

---------

Co-authored-by: SnehaRH <sneha@navadhiti.com>
Co-authored-by: SnehaRH <77656297+snehar-nd@users.noreply.github.com>
Co-authored-by: Arpit Tripathi <arpittripathiayo@gmail.com>
Co-authored-by: Vishwanath Balkur <118195001+vishwab1@users.noreply.github.com>

* Adding s3 dependency

* Making improvements given by coderabbit

* Made changes in pom release

* Use vanID instead of villageId in diagnostic document S3 path (#197)

The S3 key was prefixed with a villageId taken from the request param,
so every document in a push run was filed under one village regardless
of where it came from. Use each row's own vanID instead; villageId is
now optional on /diagnostic-documents-to-server and ignored.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* Vb/diagnostic document sync response (#198)

* Use vanID instead of villageId in diagnostic document S3 path

The S3 key was prefixed with a villageId taken from the request param,
so every document in a push run was filed under one village regardless
of where it came from. Use each row's own vanID instead; villageId is
now optional on /diagnostic-documents-to-server and ignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Return van-to-server style summary from diagnostic document push

The push always answered "Data successfully synced" when at least one
document went through, hiding partial failures. Return the same
table-level shape /van-to-server uses (status success/partial/failed,
totalRecords, successfulRecords, failedRecords, failureReasons) so the
data sync screen can show what failed and why.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* fix: update the file extension

* feat(STOP-457): automatic BenID download for StopTB laptops

The laptop now downloads BeneficiaryIDs from central by itself, so StopTB
registration no longer depends on someone clicking "Generate BenID".

- CentralAuthService: after the operator's normal login, signs in to central
  once as an API client and keeps only the refresh token (AES-GCM encrypted,
  key tied to this machine, file 600 under /opt/amrit/data). Each run uses
  refreshToken -> JWT -> getLoginResponse to get a central session key. No
  password is stored; a password central rejected is never sent again.
- BenIdAutoTopUpService: pool 0 -> 10000, below 2000 -> 1000. IDs received
  from central are written to a pending file before the local import, so a
  failed import is retried with the same IDs. Retries 5/15/30/60 min; one
  lock shared with the manual "Generate BenID" button.
- /centralAuth/status and /centralAuth/capture for MMU-UI.
- Top-up check also runs after a successful van-to-server upload and master
  download.
- Only active when stoptb.enforce.vanid=true and stoptb.van.id>0; every
  setting has a default, so other deployments need no config change.

No change on central (Common-API / BenGen).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(STOP-457): harden automatic BenID download for many laptops

- Ask central for at most 1000 IDs per request (benid.pool.maxPerRequest):
  a 10000-ID request can outlast the central gateway timeout, and every
  retry after a lost reply reserved another 10000 unused IDs.
- Remember a central-rejected laptop password across restarts (marker file
  with the user name only), so daily restarts cannot slowly lock the
  central account; a successful Data Sync reconnect clears it.
- Write the pending-import file atomically (temp + move); a pending file
  that still fails after 3 tries is set aside so top-ups never get stuck.
- Spread retry waits by +/-20% so laptops do not all hit central at once
  after an outage.
- Run background work on the feature's own single thread instead of the
  shared ForkJoin pool.

Tests: 26 unit tests pass; checked in the running app (2500 IDs arrived as
1000+1000+500; rejected login not resent after restart).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: set MMU-API version to 3.12.0

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* StopTB camp laptop data sync – = sync, inventory upload fixes and master download (#201)

* feat(AMM-2491): automatic data sync and sync failure review for StopTB laptops

- Upload (van to central) every 30 min and master download every 6 h,
  using the central connection of the automatic BenID download (no stored
  password; SyncedBy = operator who connected the laptop). Own thread and
  lock; skips while a manual master download runs.
- GET /dataSyncActivity/autoSyncStatus: last runs, last error, and rows
  central rejected (Processed='F') per table and SyncFailureReason.
- Active only where stoptb.enforce.vanid=true and stoptb.van.id>0; manual
  Upload / Download Masters unchanged. 6 unit tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(AMM-2491): send sync failure time as text

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(AMM-2491): automatic inventory data download after each upload

Refills and stock issued at central for this van's store (indents, indent
issues, stock transfers, stock entries) reach the laptop without pressing
Inventory Data Download. Runs after each automatic upload; its own failure
does not undo the upload; shown in autoSyncStatus. Switch:
sync.autoInventoryDownload.enabled (default true). 2 new unit tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(datasync): inventory data download skipped indent orders and stock transfers

The loop over the 5 tables also did i++ at the end of each pass, so only
t_indent, t_indentissue and t_itemstockentry were downloaded;
t_indentorder and t_stocktransfer never reached the van.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(datasync): central keeps the van row's VanSerialNo and store on upload

m_synctabledetail maps van columns to server columns by position (the van's
own ID sits where the server list has VanSerialNo); since the by-name record
mapping, VanSerialNo was stored NULL, so a re-sent row was inserted again
(duplicates) and joins on VanSerialNo broke. VanSerialNo now takes the van
row ID and SyncFacilityID the row's store when not sent by name
(dataSync.fillVanKeys, default true).

Master download on central logs the row count instead of every row (the
661k-row village table exhausted the heap).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(AMM-2491): StopTB laptop master download and automatic sync off by default

- StopTB laptops (stoptb.enforce.vanid=true): service-line and van masters
  (hubs, vans, stores, items, item-store mapping, roles) are downloaded in full
  for the van's line, so a laptop set up from a copied database gets them all;
  shared masters (villages) stay incremental. Other MMU laptops unchanged.
- Automatic upload / master download / inventory download are off by default
  (manual sync until approved); when on, the automatic master download fetches
  only the inventory masters (sync.autoMasterDownload.inventoryOnly=true).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(datasync): central links uploaded rows to central visit/prescription IDs

A van sends its own IDs in BenVisitID / PrescriptionID; on central those
numbers belong to other rows, so prescriptions and prescribed drugs were
linked to another van's visit/prescription, or rejected by the foreign key
when the number did not exist. With dataSync.translateVanLinks=true the link
is replaced by the central ID of the row the same van uploaded with that
VanSerialNo; if that row is not on central yet the record is not saved and is
sent again on the next upload. Off by default (other MMU servers unchanged);
to be switched on for the StopTB central.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(datasync): StopTB laptop uploads stock tables only through the Inventory group

Inventory tables are configured in group 5 (old lists: stock-outs without
FacilityID/SyncFacilityID, patient issues without SyncFacilityID, batch insert
fails) and in group 9 Inventory (complete). Group 5 runs first and marks the
rows uploaded, so central stored stock-outs without their camp store and
per-store reports missed them. On StopTB laptops (stoptb.enforce.vanid=true) a
table that a later group also uploads is skipped. Other MMU laptops unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(datasync): StopTB laptop upload uses the renamed reason_to_close column and continues past a failing table

The laptop sync list still names tb_diagnostic_order.reason_for_refusal; V125 renamed it to
reason_to_close and the whole upload stopped with bad SQL grammar. The column name the table
really has is used for the van and server column lists, and a table that cannot be read is
recorded as failed while the other tables still upload. Both only on a StopTB laptop
(stoptb.enforce.vanid).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* fix(downsync): create worklist rows for down-synced beneficiaries (#203)

* fix(downsync): create worklist rows for down-synced beneficiaries

Down-sync copies the beneficiary tables but not i_ben_flow_outreach, and
FLW-API getBeneficiaryData serves tabs only beneficiaries that have a
worklist row. Beneficiaries registered elsewhere and down-synced to the
laptop therefore never reached the laptop's other tabs, so tab counts
differed.

After each down-sync, insert the missing worklist row for this van's
beneficiaries, filled as TM-API does at registration. Processed = 'P' so
it is not uploaded back (central already has its own row); modified_by =
'downsync' marks these rows. A failure here is logged and does not undo
the down-sync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(s3): let vans start without AWS keys — only central needs them (#202)

Vans (camp laptops) run the same war as central but never call S3: they
push diagnostic documents to central over HTTP. Yet startup failed with
"Could not resolve placeholder 'AWS_S3_ACCESS_KEY'", and blank values
would still crash in AwsBasicCredentials.create ("Access key ID cannot be
blank").

- common_docker.properties: default the S3 placeholders (keys/bucket
  empty, region ap-south-1, push batch size 3).
- S3ClientConfig: with blank keys use DefaultCredentialsProvider, which
  resolves lazily on first S3 call instead of at startup.

Central keeps setting real AWS_S3_* values and behaves as before.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 10b8ffa)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: vanitha1822 <vanitha@navadhiti.com>
Co-authored-by: Vishwanath Balkur <118195001+vishwab1@users.noreply.github.com>
Co-authored-by: vishwab1 <vishwanath@navadhiti.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Saurav Mishra <80103738+SauravBizbRolly@users.noreply.github.com>
Co-authored-by: SnehaRH <77656297+snehar-nd@users.noreply.github.com>
Co-authored-by: Sneha <sneha@ADMINs-MacBook-Pro.local>
Co-authored-by: Vanitha S <116701245+vanitha1822@users.noreply.github.com>
Co-authored-by: SnehaRH <sneha@navadhiti.com>
Co-authored-by: Arpit Tripathi <arpittripathiayo@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants