Created cardinal Semiotic class for Kannada TN - #448
Conversation
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
…iles Signed-off-by: richa-2002 <richa@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
9149d5a to
53b3447
Compare
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
|
This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days. |
for more information, see https://pre-commit.ci
|
Thank you for the review, @mgrafu. I have addressed all the changes you suggested. Additionally, I removed the unnecessary weights from cardinal.py file from tagger module. Testing |
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: richa-2002 <richa@nvidia.com>
Signed-off-by: richa-2002 <richa@nvidia.com>
for more information, see https://pre-commit.ci
|
Thank you for the review, @mgrafu. I have addressed all the changes you suggested. Testing
|
| import pynini | ||
| from pynini import Far | ||
| from pynini.export import export | ||
| from pynini.lib import byte, pynutil, utf8 |
There was a problem hiding this comment.
please clean up any unused imports in all scripts (byte is not used here)
There was a problem hiding this comment.
please check other imports / files
There was a problem hiding this comment.
Done , removed unused imports from utils.py file.
Signed-off-by: richa-2002 <richa@nvidia.com>
|
Thank you for the review, @mgrafu. I have addressed all the changes you suggested. Testing Pytest: All test cases passed — Cardinal, Word, and Punctuation. |
| ನಾನು 20' ಮತ್ತು 14/ ಅವರು ನಾನು 16.~ನಾನು ಇಪ್ಪತ್ತು ' ಮತ್ತು ಹದಿನಾಲ್ಕು / ಅವರು ನಾನು ಹದಿನಾರು . | ||
| ಅವನು ಮತ್ತು ನಾನು ಮತ್ತು ಮುಲ್ಲರ್ನ ಮತ್ತು ದೆತ್ರ್~ಅವನು ಮತ್ತು ನಾನು ಮತ್ತು ಮುಲ್ಲರ್ನ ಮತ್ತು ದೆತ್ರ್ | ||
| 'ಪರೀಕ್ಷೆ'.~'ಪರೀಕ್ಷೆ'. | ||
| ಇದು (12 ಪ್ರಭೇದಗಳು) ಮತ್ತು...~ಇದು ( ಹನ್ನೆರಡು ಪ್ರಭೇದಗಳು) ಮತ್ತು... |
There was a problem hiding this comment.
why are spaces added for some parens but not others?
There was a problem hiding this comment.
For a phrase like: “ಇದು (12 ಪ್ರಭೇದಗಳು)”
the the tokenizer sees :“(”, number, word, “)” , all of them as separate token but the kannada grammar is not forcing the paranthesis to keep a space for the closing bracket for the word.
so it is becoming:“( ಹನ್ನೆರಡು ಪ್ರಭೇದಗಳು)”
There is only one internal space after the opening parenthesis, and none before the closing one, because the closing bracket is right after the word.
For the test case like:“(1), (2), (3)”
each item is treating as an independent cardinal wrapped by the punctuation:“( ಒಂದು )”, “( ಎರಡು )”, “( ಮೂರು )”
so the output intentionally keeps spaces on both sides inside each parenthesis.
| 1! ಹಲೋ~ಒಂದು! ಹಲೋ | ||
| 1 ,~ಒಂದು , | ||
| (1) ಹಲೋ~(ಒಂದು) ಹಲೋ | ||
| 1! ಹಲೋ~ಒಂದು! ಹಲೋ |
There was a problem hiding this comment.
why is this test repeated?
There was a problem hiding this comment.
please remove repeated test cases in all data files
Signed-off-by: richa-2002 <richa@nvidia.com>
|
Thank you for the review, @mgrafu. I have addressed all the changes you suggested. Testing Pytest: All test cases passed — Cardinal, Word, and Punctuation. |
3 participants
What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
Before your PR is "Ready for review"
Pre checks:
git commit -sto sign.pytestor (if your machine does not have GPU)pytest --cpufrom the root folder (given you marked your test cases accordingly@pytest.mark.run_only_on('CPU')).bash tools/text_processing_deployment/export_grammars.sh --MODE=test ...pytestand Sparrowhawk here.__init__.pyfor every folder and subfolder, includingdatafolder which has .TSV files?Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.to all newly added Python files?Copyright 2015 and onwards Google, Inc.. See an example here.try import: ... except: ...) if not already done.PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.