feat: add POS tag sync workflow with C++ upstream (closes #40) - #46
Merged
Conversation
Member
Author
|
@kkweon 넵 부탁드리겠습니다! |
The generated postype.go did not compile: POS_P, POS_PV and POS_PA were
all emitted as "P", which produced duplicate cases in the isValid switch.
Rewrite the generator to evaluate POSTag enum initializers the way the
C++ compiler does and to transcribe tagToString/tagRToString, instead of
relying on hardcoded special cases and an "ends with i" heuristic. That
corrects three mappings that never matched upstream:
- pa = p + 1 == max, so tagToString returns "@", not "P"
- pvi/pai fall through the irregular switch default, so both are "@",
not "PV-I"/"PA-I"
- the -R variants are reachable only through tagRToString, which is
what kiwi_res_tag calls; dropping them made ParsePOSType fail on
ordinary sentences such as "편지를 받았다", re-introducing #39
Deduplicate the isValid switch by value so aliases can no longer break
the build, keep POS_USER_0..4 as deprecated aliases so the rename does
not break downstream code, and add a regression test covering both the
-R tags and a regular conjugation through Analyze.
Verify the generated code inside the sync workflow: pull requests opened
with GITHUB_TOKEN do not trigger CI, so nothing was checking it.
ParsePOSType is the Go counterpart of upstream's toPOSTag, so derive the accepted strings from that function instead of inventing them. The previous commit mapped POS_PA, POS_PV_I and POS_PA_I to "@". That string is not something upstream ever converts to or from: tagToString guards its table with assert(t < POSTag::max) and pa == max, so reading "@" only happens in an NDEBUG build past the assertion, and toPOSTag has no entry for "@" at all. Drop the three constants rather than exposing a sentinel as a tag. Restore POS_V and add the remaining strings toPOSTag accepts but Kiwi never emits: A (POSTag::p), NF, NV, NA and UNK (POSTag::unknown), and ^ (POSTag::unknown, spelled POS_CARET). POS_V had been removed on the grounds that Kiwi never returns "V", which is true of tagToString but irrelevant to a parser -- toPOSTag accepts it. kiwigo now accepts exactly toPOSTag's 75 strings plus "P", which tagToString can return for POSTag::p even though toPOSTag does not accept it. Emit the two groups as separate blocks so the generated alignment matches gofmt.
kkweon
force-pushed
the
feat/pos-tag-sync-workflow
branch
from
August 12, 2026 00:33
49deb8b to
6c5f289
Compare
kkweon
approved these changes
Aug 12, 2026
kkweon
left a comment
kkweon
left a comment
Member
There was a problem hiding this comment.
원래 커밋은 main 에 이미 반영되어 있어서(0ff8d45), 중복을 제거하고 이 브랜치에는 후속 수정만 남겼습니다.
수정 내용:
- 생성기가
POSTagenum 값을 직접 평가하고tagToString/tagRToString/toPOSTag를 그대로 전사하도록 변경 isValid를 값 기준으로 중복 제거 (duplicate case빌드 실패 해소)-R태그 복구 (#39 재발 방지) + 회귀 테스트 추가- 동기화 워크플로에 빌드/테스트 게이트 추가 — GITHUB_TOKEN 으로 생성된 PR 은 CI 가 돌지 않아 검증 공백이 있었습니다
CI 통과했습니다. 머지하면 현재 실패 중인 main 도 함께 복구됩니다.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
POS Tag Sync Workflow Implementation
C++ Kiwi 원본 저장소의 POS 태그 정의와 Go 코드를 자동으로 동기화하는 워크플로우를 구현합니다.
Changes
1. 추출 스크립트
scripts/extract_postags.py2. GitHub Actions 워크플로우
.github/workflows/sync-postypes.yaml3. Makefile 타겟
4. postype.go 업데이트
POS_P,POS_PV,POS_PA,POS_PV_I,POS_PA_IPOS_V,POS_VV_R,POS_VA_R,POS_VX_R,POS_XSA_RPOS_USER_0→POS_USER0Verification
Closes #40