Skip to content

[zscaler_zia] fix timestamp parsing - #21473

Open
brijesh-elastic wants to merge 3 commits into
elastic:mainfrom
brijesh-elastic:zscaler_zia-4.4.1
Open

brijesh-elastic wants to merge 3 commits into
elastic:mainfrom
brijesh-elastic:zscaler_zia-4.4.1

Conversation

@brijesh-elastic

@brijesh-elastic brijesh-elastic commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Proposed commit message

zscaler_zia: fix timestamp parsing for NSS feeds emitting yyyy-MM-dd HH:mm:ss

The date processors in the Audit, DNS, Email DLP, Endpoint DLP, Firewall,
SaaS Security, and Tunnel data streams listed the fallback format as
`yyyy-mm-dd HH:mm:ss`, where lowercase `mm` means minute-of-hour rather than
month. NSS feeds customized to emit numeric timestamps therefore failed to
parse and the events landed with a date parse error. Correct the format to
`yyyy-MM-dd HH:mm:ss` in all seven pipelines and add an Email DLP pipeline
test record using the numeric timestamp shape.

Closes #​19729

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

  • Clone integrations repo.
  • Install elastic package locally.
  • Start elastic stack using elastic-package.
  • Move to integrations/packages/zscaler_zia directory.
  • Run the following command to run tests.

elastic-package test

Related issues

@brijesh-elastic brijesh-elastic self-assigned this Sep 21, 2026
@brijesh-elastic
brijesh-elastic requested a review from a team as a code owner September 21, 2026 06:45
@brijesh-elastic brijesh-elastic added the Integration:zscaler_zia Zscaler Internet Access label Sep 21, 2026
@brijesh-elastic
brijesh-elastic requested a review from a team as a code owner September 21, 2026 06:45
@brijesh-elastic brijesh-elastic added bugfix Pull request that fixes a bug issue Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] Team:SDE-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors] labels Sep 21, 2026
@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@github-actions

Copy link
Copy Markdown
Contributor

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@efd6 efd6 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM when the additional tests are in place.

@vera-review-bot

Copy link
Copy Markdown

🟢 No issues across the latest commits b07a247.

Review summary

Issues found across earlier commits a08dfad — 1 medium, 1 low

Package-level:

  • 🟡 Only the email_dlp fixture exercises the new yyyy-MM-dd HH:mm:ss format

    Append one event per affected fixture with the customized NSS timestamp shape, e.g. for the tunnel stream:

    {"sourcetype":"zscalernss-tunnel","event":{"datetime":"2026-06-24 12:34:56","Recordtype":"Tunnel Event","tunneltype":"IPSEC IKEV 2","user":"user@​example.com","location":"Example Location","sourceip":"203.0.113.10","destinationip":"198.51.100.20","sourceport":"500","event":"Tunnel Up","eventreason":"None","recordid":"1972900000000000002"}}

    Repeat with the equivalent time field (time, datetime, eventtime/feedtime, bypass_time, last_modified_time/last_shared_on) for each of the other five streams and regenerate the expected outputs.

  • 🔵 Proposed commit message

    zscaler_zia: fix timestamp parsing for NSS feeds emitting yyyy-MM-dd HH:mm:ss
    
    The date processors in the Audit, DNS, Email DLP, Endpoint DLP, Firewall,
    SaaS Security, and Tunnel data streams listed the fallback format as
    `yyyy-mm-dd HH:mm:ss`, where lowercase `mm` means minute-of-hour rather than
    month. NSS feeds customized to emit numeric timestamps therefore failed to
    parse and the events landed with a date parse error. Correct the format to
    `yyyy-MM-dd HH:mm:ss` in all seven pipelines and add an Email DLP pipeline
    test record using the numeric timestamp shape.
    
    Closes #​19729
    

A new commit triggers another review — at most once every 15 minutes. I skip the PR while it's approved or has merge conflicts.

🤖 AI-Generated Review | Vera Review Bot - v0.4.2 | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

✅ All changelog entries have the correct PR link.

@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

💚 Build Succeeded

History

cc @brijesh-elastic

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Pull request that fixes a bug issue Integration:zscaler_zia Zscaler Internet Access Team:SDE-Crest Crest developers on the Security Integrations team [elastic/sit-crest-contractors] Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug-hunter] zscaler_zia Email DLP rejects numeric timestamp format

2 participants