Compare commits

..

428 Commits

Author SHA1 Message Date
Cedric Staniewski 343d003f7a conf: Update version to 0.14
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/206
2016-07-17 12:34:58 +00:00
Frank Sachsenheim 5738ea240d users/config: Correct username element -> user
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/205
2016-07-17 06:51:32 +00:00
Jakob Borg 57aa23d29c index: discosrv & relaysrv were renamed 2016-07-05 15:00:15 +02:00
Jakob Borg 22fea7df4e specs: Local discovery version was bumped 2016-07-05 14:58:24 +02:00
Jakob Borg 77f8d085e5 deploy: Script must have bash header 2016-07-05 14:56:12 +02:00
aviau aae1a525f1 users: create stdiscosrv manpage
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/203
2016-07-04 14:28:49 +00:00
Jakob Borg 76236e1797 specs: Document instance_id in local discovery 2016-07-04 15:23:18 +02:00
Jakob Borg 91818c9890 dev/issues: Clarify labels and add milestone info
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/199
2016-07-04 13:18:43 +00:00
aviau a1ac0aa446 users: create strelaysrv manpage
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/202
2016-07-04 13:17:20 +00:00
Jakob Borg a9b3fef697 specs: Update specs for protocol buffer formats
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/190
2016-07-04 11:56:58 +00:00
Jakob Borg 3b76ff6329 users/config: Empty is also an option for folder labels 2016-06-30 13:31:09 +02:00
Cedric Staniewski 6307a1b1db users/config: Folder label are not unique
See https://github.com/syncthing/syncthing/issues/3367

GitHub-Pull-Request: https://github.com/syncthing/docs/pull/200
2016-06-30 11:29:55 +00:00
aviau f1808d4aa2 typo: interchangably -> interchangeably
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/198
2016-06-29 13:41:59 +00:00
Peter Dave Hello 2a48e397da optimize png images using zopflipng
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/197
2016-06-29 13:40:51 +00:00
Cedric Staniewski 60c7245e44 Update syncthing major version
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/195
2016-06-27 20:55:33 +00:00
Cedric Staniewski 43bae4005c rest/system/ping: Remove obsolete note
A GET request to the /rest/system/ping endpoint also requires an API key or CSRF token.

GitHub-Pull-Request: https://github.com/syncthing/docs/pull/194
2016-06-27 20:47:00 +00:00
Alexander Graf edf2498b76 rest: Document override call (fixes #39)
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/193
2016-06-26 09:20:24 +00:00
twomice 4a4566570e spellcheck "remove" => "remote"
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/192
2016-06-17 08:52:08 +00:00
Jerry Jacobs 44553d2048 Mention and deprecate user-contrib OS X application (fixes #188)
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/189
2016-06-15 16:38:23 +00:00
Jakob Borg 8241da8264 Note -gui-apikey as deprecated (fixes #186) 2016-06-08 14:16:10 +02:00
Zillode cbdc571b8c rest/system-status-get.rst: added new fields
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/187
2016-06-07 05:50:21 +00:00
imsodin c6b994a815 Update /rest/db/ignores for 0.13 changes
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/174
2016-06-03 13:09:36 +00:00
uglygus 8748339201 (?d) example
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/182
2016-06-03 11:07:17 +00:00
Jakob Borg 899b9fc6c7 dev/mergebot: Emphasize the importance of getting the "Skip-check: authors" part right
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/185
2016-06-03 10:32:08 +00:00
Majed Abdulaziz 88f88a948d Document lastScan field in /rest/stats/folder API endpoint
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/183
2016-06-03 09:58:38 +00:00
Jakob Borg 936cef6ddd dev/mergebot: Describe LGTM flow
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/184
LGTM: AudriusButkevicius, rumpelsepp
2016-06-03 09:37:49 +00:00
Zillode 3b286b9362 docs/rest: updated connection types
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/176
2016-05-31 06:23:43 +00:00
Zillode 0f5701f175 events/deviceconnected.rst: Added new fields
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/180
2016-05-31 06:23:08 +00:00
Zillode 976870ec0e events/devicepaused.rst: document DevicePaused
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/179
2016-05-31 06:22:54 +00:00
Zillode 6ff9af7872 events/deviceconnected.rst: Added new fields
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/178
2016-05-31 06:22:34 +00:00
Zillode 2b1bcd5bfc events/folderrejected.rst: Added folderLabel field
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/181
2016-05-31 06:22:11 +00:00
Zillode 34b34c7b77 rest/system-config-get: Added exemplary data
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/177
2016-05-30 21:12:12 +00:00
Arthur Lutz 47a88349ff users/contrib: Add Saltstack contribution
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/175
2016-05-30 08:09:50 +00:00
Scott Hansen 699786cee1 users/discosrv: Update for /v2/ path component
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/173
2016-05-26 14:08:09 +00:00
Audrius Butkevicius c4a1600950 Update discovery protocol specs for v0.13
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/171
2016-05-21 02:08:46 +00:00
Audrius Butkevicius ddd8042732 Document need pagination (fixes #81)
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/172
2016-05-21 01:35:37 +00:00
Gavrilov Aleksej 28f626f6d4 users/contrib: Add ansible
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/170
2016-05-19 00:01:33 +00:00
Jakob Borg 0bef7754bb users/config: Update config article for current config format (v13)
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/169
2016-05-17 07:10:55 +00:00
Benjamin Masters fd92efc2f6 users/syncing: Minor spelling correction
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/168
2016-05-12 05:58:39 +00:00
Jakob Borg 076ac1aa14 users/faq: Mention scanning and syncthing-inotify in relation to CPU usage
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/167
2016-05-11 11:44:45 +00:00
Lars K.W. Gohlke 0231d12897 users/config: Document folder labels
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/162
2016-05-11 10:48:38 +00:00
Lars K.W. Gohlke c5796b993f readme: Getting started building docs locally
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/164
2016-05-11 10:47:49 +00:00
Jakob Borg 5ac7ff99da dev: Document using the merge bot (#166)
* dev: Document using the merge bot
2016-05-08 13:53:15 +01:00
ka7 7eacdaa11b spelling-fixes 2016-05-08 10:20:20 +01:00
Lars K.W. Gohlke bdafe9f5d4 Updated documentation for foldertype 2016-05-08 07:20:16 +02:00
Audrius Butkevicius de1d3792c1 Document temporary indexes
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/161
2016-05-01 11:21:12 +00:00
Audrius Butkevicius 69cdae0149 Document (?d) prefix
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/160
2016-04-30 18:51:14 +00:00
Audrius Butkevicius 8d245533a7 specs/bep-v1: Add Hello message, changed ClusterConfig
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/159
2016-04-30 12:44:20 +00:00
Jakob Borg 0c91ae2743 users/faq: Mention GOMAXPROCS in question about CPU usage 2016-04-29 16:49:16 +02:00
Jakob Borg 32a16746e5 users/faq: Link to firewall setup from relay question 2016-04-29 16:40:42 +02:00
Jakob Borg 630ec0cd28 users/faq: Add Why is the sync so slow? 2016-04-29 16:33:32 +02:00
Jakob Borg 09150a77ce users/faq: Update and clarify some FAQ answers 2016-04-29 15:56:18 +02:00
Jakob Borg f5ec46e66d users/syncing: Add article describing the underlying mechanisms 2016-04-29 15:34:25 +02:00
Jakob Borg 0e21c6fc0d dev/contributing: Describe new commit message and merge process 2016-04-29 12:11:58 +02:00
Jakob Borg c648d3bceb index: Logically group articles, undo renaming discosrv 2016-04-29 12:05:17 +02:00
Jakob Borg 7c8c38e0be users: Homogenize titling for Custom {Discovery, Relay, Upgrade} Server 2016-04-29 11:54:46 +02:00
Jakob Borg 43699de504 users/firewall: Link to proxy setup 2016-04-29 11:51:13 +02:00
Jakob Borg c50ade832b users/proxying: Add article describing SOCKS proxies 2016-04-29 11:34:57 +02:00
Scott Hansen 5ebce76524 users/discosrv: Document running discosrv behind nginx
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/149
2016-04-29 09:13:57 +00:00
Jakob Borg 7ad9ba5a70 users/autostart: Simplify brew instructions 2016-04-29 11:10:45 +02:00
Edd Barrett c31350e80d Link to OpenBSD port. (#157) 2016-04-21 21:08:21 +01:00
Iliyan ab0a7268f6 Deleted my app
App's not finished

GitHub-Pull-Request: https://github.com/syncthing/docs/pull/155
2016-04-18 20:20:26 +00:00
Iliyan 1a8dbc1058 Update contrib.rst
Added my "SyncthingLauncher" app

GitHub-Pull-Request: https://github.com/syncthing/docs/pull/154
2016-04-18 19:56:47 +00:00
Louis Sautier f6b4c34187 Add a link to the official Gentoo package
GitHub-Pull-Request: https://github.com/syncthing/docs/pull/151
2016-04-13 11:10:17 +00:00
Jakob Borg b8bf3411cc Merge pull request #150 from syncthing/flagdir
Add directory flag to Index message and correct bit assignments
2016-04-10 11:47:37 +01:00
djtm cc5746632a docs: contrib.rst/docker: add syncthing-docker-scratch 2016-04-06 15:48:38 +02:00
ProactiveServices 36d399c3cb Windows service/NSSM improvements
Cleaned up style, grammar and added steps to enable logging and support
Syncthing exit/restart/upgrade actions. Additional note regarding
security concerns. Removed superfluous final paragraph. Somewhat more
modern and tidier screenshot, renamed it to make it less generic
2016-03-30 11:18:41 +00:00
Laurent Etiemble f27f6dff3c Indent URL under unofficial CentOS packages 2016-03-30 11:17:39 +00:00
Adam Piggott cb62b52a30 Windows does not support escaping (ref #2822) 2016-03-30 11:15:54 +00:00
Martin Freund dec9d4c26d Added Windows example for external file versioning
The example uses native batch scripting and is based on the already provided Unix example.
2016-03-30 11:14:44 +00:00
Audrius Butkevicius bd3ce4a771 Merge pull request #145 from sodacode/patch-1
Add a Windows installer
2016-03-30 01:07:41 +01:00
Audrius Butkevicius 28da8242b2 Merge pull request #138 from lkwg82/master
updated spec ...
2016-03-28 15:58:27 +01:00
Dakota abf0124e28 Update contrib.rst
Not sure why nobody has done this yet, maybe I'm missing something?
Wrote a small script for a Windows installer.
2016-03-27 10:34:11 -04:00
Audrius Butkevicius 6b2cddc6d6 Merge pull request #144 from vhbit/patch-1
Fix docs for join session flow
2016-03-26 13:37:09 +00:00
Valerii Hiora 186b96be64 Fix docs for join session flow
`JoinSessionRequest` sounds more appropriate and actually works.
2016-03-26 09:05:32 +02:00
Audrius Butkevicius 518dffae28 Merge pull request #143 from lkwg82/patch-1
Update syncthing.rst
2016-03-25 22:25:21 +00:00
Lars K.W. Gohlke 7fd012b875 Update syncthing.rst
fixes #102
2016-03-25 23:10:04 +01:00
Lars K.W. Gohlke e36eacb046 Update bep-v1.rst 2016-03-25 22:56:12 +01:00
Audrius Butkevicius c06e202121 Merge pull request #142 from lkwg82/patch-3
Update syncthing.rst
2016-03-25 18:03:00 +00:00
Lars K.W. Gohlke 0ddcafa6c7 Update syncthing.rst
fixes #15
2016-03-25 18:53:08 +01:00
Audrius Butkevicius 67b1a291c6 Merge pull request #140 from lkwg82/patch-1
Update syncthing.rst
2016-03-25 17:41:11 +00:00
Lars K.W. Gohlke 65c2a85971 Update syncthing.rst
added 

-browser-only
-paused

fixes #95
2016-03-25 18:39:37 +01:00
Audrius Butkevicius 82fb189d1d Merge pull request #103 from tuathail/patch-1
autostart.rst Mac OS X Updates
2016-03-25 17:34:30 +00:00
Lars K.W. Gohlke c260c16cf5 updated spec according to syncthing/syncthing/pull/2668 2016-03-25 18:11:50 +01:00
Audrius Butkevicius 9adb78c4a6 Merge pull request #137 from syncthing/ProactiveServices-docs-PR2880
Document use of tcp4 and tcp6
2016-03-25 13:23:05 +00:00
Adam Piggott 7c5b00deb1 Add GUI Element address examples 2016-03-25 10:15:42 +00:00
Adam Piggott 609a95318e Document tcp4:// tcp6:// prefixes
Documenting https://github.com/syncthing/syncthing/pull/2880
2016-03-25 09:46:47 +00:00
Jakob Borg c0820c8a0c Delete followsymlinks.rst
For now, as it causes endless confusion.
2016-03-24 16:46:50 +01:00
Jakob Borg 279fd90d57 Update release process 2016-03-23 10:21:21 +01:00
Jakob Borg be75b6c28a Merge pull request #136 from kluppy/patch-4
Clarify remote states when ignoreDelete selected
2016-03-21 12:33:09 +01:00
Scott Klupfel 1941ddf320 Clarify remote states when ignoreDelete selected
As discussed on the forum: https://forum.syncthing.net/t/stuck-syncing-at-90/6991
2016-03-21 19:29:31 +10:00
Jakob Borg 616a1d5d3c Merge pull request #135 from wweich/patch-1
Correct unit for rate limit (fixes syncthing/syncthing#2844)
2016-03-17 07:54:18 +01:00
Wulf Weich 0cd687b714 Correct unit for rate limit (fixes syncthing/syncthing#2844)
Correct unit for `maxSendKbps` and `maxRecvKbps` from kibibits to kibibytes.
2016-03-17 06:13:05 +01:00
Jakob Borg c2a514250f Infrastructure updates 2016-03-14 08:46:33 +01:00
Audrius Butkevicius eb681e3252 Merge pull request #133 from syncthing/revert-132-patch-1
Revert "adjust path to theme path"
2016-03-09 13:04:52 +00:00
Audrius Butkevicius 0bdfaabceb Revert "adjust path to theme path" 2016-03-09 13:04:42 +00:00
Audrius Butkevicius 218001c902 Merge pull request #132 from lkwg82/patch-1
adjust path to theme path
2016-03-09 00:37:28 +00:00
Lars K.W. Gohlke d2a7faed49 adjust path to theme path 2016-03-08 22:39:00 +01:00
Jakob Borg 7079b3f0c9 Add note about compiler versions and vendoring 2016-03-08 08:36:05 +01:00
Jakob Borg b620e636cd Merge pull request #131 from syncthing/revert-130-patch-2
Revert "updated compiler version"
2016-03-08 08:29:00 +01:00
Jakob Borg 4883e834e9 Revert "updated compiler version" 2016-03-08 08:28:50 +01:00
Audrius Butkevicius abecfcd1b1 Merge pull request #130 from lkwg82/patch-2
updated compiler version
2016-03-07 22:14:06 +00:00
Lars K.W. Gohlke 0a251931cb updated compiler version
see compiler change to version 1.6 in https://github.com/syncthing/syncthing/releases/tag/v0.12.20
2016-03-07 22:55:34 +01:00
Audrius Butkevicius 8ee999962e Merge pull request #128 from bugith/patch-1
Update autostart.rst
2016-03-04 20:29:36 +00:00
bugith 7a74caea05 Update autostart.rst
AFAIK the Windows startup folder is parsed on user login, not before.
2016-03-04 21:17:30 +01:00
Jakob Borg 72ab46b379 Merge pull request #127 from letiemble/patch-2
Companion PR for the syncthing/syncthing#2791
2016-02-28 20:06:18 +07:00
Laurent Etiemble cef5fd6282 Companion PR for the syncthing/syncthing#2791
Modification of the specifications to match this [pull request](https://github.com/syncthing/syncthing/pull/2791)
2016-02-28 12:21:41 +01:00
Stefan Tatschner 7c4f1597e2 Add semicolons to XDR code
This ensures that the documented XDR code compiles using
rpcgen.
2016-02-23 12:39:43 +01:00
Audrius Butkevicius 2a4b5025a5 Merge pull request #125 from dinosore/patch-1
spelling
2016-02-20 19:02:55 +00:00
dinosore b108d35f5d spelling 2016-02-20 18:52:56 +00:00
Stefan Tatschner 648e3b77d7 Merge pull request #124 from MingweiSamuel/patch-1
documentation replace "|" with "+" for box
2016-02-19 08:25:25 +01:00
Mingwei Samuel 552d4e0a32 Update bep-v1.rst 2016-02-18 21:57:19 -08:00
Audrius Butkevicius 5873e3d3f7 Merge pull request #119 from syncthing/Proactive-Services-faq-localsync
Add FAQ for syncing folders on same system
2016-02-09 15:13:31 +00:00
ProactiveServices d159b1ec9a Fixed underline 2016-02-09 14:38:09 +00:00
ProactiveServices dec3abaaf7 Add FAQ for syncing between folders on same system 2016-02-09 13:22:41 +00:00
Audrius Butkevicius bb7a50ea03 Merge pull request #118 from syncthing/Proactive-Services-pathdocs
Document -paths
2016-02-08 22:28:28 +00:00
ProactiveServices aabc64b257 Document -paths
Document -paths parameter (fixes #116)
2016-02-08 22:15:50 +00:00
Audrius Butkevicius 2adef7ebfb Merge pull request #117 from mlazarov/patch-1
Adding CentOS package
2016-02-08 19:53:29 +00:00
Martin Lazarov 5abe956ebe Update contrib.rst 2016-02-08 19:47:26 +02:00
Martin Lazarov b92361e45e Adding CentoOS package 2016-02-08 19:39:06 +02:00
Jakob Borg 708ea1f84a Merge pull request #115 from syncthing/ProactiveServices-device-docs-prefixes
Fix device config re: prefixes
2016-02-08 02:05:37 -05:00
Adam Piggott 544b7935fd Update config.rst
Re-added line that was unintentionally removed. Further clarify detail before example section.
2016-02-08 03:43:21 +00:00
Adam Piggott b0c16a631c Fix device config re: prefixes
Added tcp:// prefixes in docs and examples, added host name detail, fixes to IPv6 examples.
2016-02-08 02:16:39 +00:00
Audrius Butkevicius a08d09fca3 Merge pull request #113 from Morphy99/patch-1
Update autostart.rst
2016-02-01 15:21:45 +00:00
Morphy99 51c935d6e1 Update autostart.rst
Added configuration directory location under windows service method as this differs to other methods.
2016-02-01 15:16:15 +00:00
Jakob Borg 845bc0363c Merge pull request #111 from camsteffen/master
add -no-console -no-browser options to Task Scheduler
2016-01-30 09:49:38 +01:00
Jakob Borg 4e9c5102bb Remove docker instructions 2016-01-30 09:14:40 +01:00
Cameron Steffen 59423ddf86 add -no-console -no-browser options to Task Scheduler 2016-01-29 23:53:01 -06:00
simplypeachy 3b25e2ed2f Fix error in unprivileged user section
Thanks to emdete on IRC for pointing out the error.
2016-01-24 16:50:58 +00:00
Jakob Borg 4a792c572e Merge pull request #108 from simplypeachy/patch-1
Update config.rst to include ignoreDevice element
2016-01-19 15:16:15 -08:00
simplypeachy f02eadc886 Update config.rst to include ignoreDevice element 2016-01-19 23:07:24 +00:00
Jakob Borg 1a2f99efe8 Correct internal->lib in dev intro 2016-01-19 14:14:28 -08:00
Stefan Tatschner 3a5ba9f822 Merge pull request #107 from ojongerius/patch-2
Remove links which are not visible to non team members.
2016-01-17 14:23:09 +01:00
Otto Jongerius 6f97972956 Remove links
They are either dead or not visible to non team members.
2016-01-17 23:28:14 +11:00
Audrius Butkevicius 74eb5e2fec Merge pull request #106 from ojongerius/patch-1
Correct example
2016-01-17 00:59:46 +00:00
Otto Jongerius a0e1f38fe2 Correct example 2016-01-17 11:51:50 +11:00
Jakob Borg 4d1c1143c2 Retract hash algo proposal for the moment 2016-01-12 12:01:07 +01:00
Jakob Borg e59d845978 Correct statechanged event states 2016-01-12 11:59:39 +01:00
tuathail 7c3ee6f441 autostart.rst Mac OS X Updates
Some clarifications to the Mac OS X install including:
Spelled out the home directory (/home/<username>/bin) in step one. Some people might gloss over it and just put it in /bin otherwise.
4 replacements of username are required, not 2.
Replaced the ~ with /Users/<username> for consistency, but also if the user switched to root user to avoid it being copied to /var/root/Library.. instead of /Users/<username>/Library...
2016-01-11 20:52:00 +00:00
Audrius Butkevicius f16ab3bdcf Merge pull request #101 from kluppy/master
Clarify Direct Connection Prefered Over Relay
2016-01-07 11:11:40 +00:00
Scott Klupfel 1694117003 Clarify Direct Connection Prefered Over Relay 2016-01-07 21:02:11 +10:00
Scott Klupfel e3eaa4110e Merge pull request #1 from syncthing/master
Bringing Up to Date
2016-01-07 20:51:22 +10:00
Audrius Butkevicius 8ad18f22b0 Merge pull request #99 from kluppy/patch-2
Add description for progressUpdateIntervalS
2016-01-06 08:37:48 +00:00
Scott Klupfel 4175c3a93c Reverted kibibits change
Sorry, my bad.
2016-01-06 15:15:33 +10:00
Scott Klupfel dbd2a292e4 Add description for progressUpdateIntervalS & tidy 2016-01-06 15:08:34 +10:00
Jakob Borg bb87d0f18c Merge pull request #78 from syncthing/hashalgos
Add hash algos as flags on the folder in cluster config
2016-01-01 21:48:37 +01:00
Jakob Borg 2eea0f9799 Add hash algos as flags on the folder in cluster config 2016-01-01 21:47:26 +01:00
Jakob Borg be0715676c Merge pull request #98 from jimmyjones2/patch-1
Support space in directory
2015-12-30 15:42:32 +01:00
Jimmy Jones 0a72691cc4 Support space in directory
Currently the script breaks if versionspath or filepath contain a space, this fixes it.
2015-12-30 14:34:12 +00:00
Audrius Butkevicius bf2eb524f4 Merge pull request #97 from mkantor/patch-1
Syncthing is not beta anymore in SynoCommunity.
2015-12-28 02:14:55 +00:00
Matt Kantor dc1b6d3ecc Syncthing is not beta anymore in SynoCommunity.
The Syncthing package is not marked as beta these days, so there is no 
need to change the package manager settings.
2015-12-27 17:57:40 -08:00
Jakob Borg 0cfbceb428 Add other build servers 2015-12-21 12:04:47 +01:00
Jakob Borg 203ff4a7b1 Merge branch 'pr/96'
* pr/96:
  Add note about running syncthing instance on discovery server
2015-12-18 15:32:32 +01:00
Scott Klupfel c7795aa858 Add note about running syncthing instance on discovery server 2015-12-18 15:32:03 +01:00
Jakob Borg 008e5a0ac0 Merge pull request #93 from sbraz/master
Fix typos
2015-12-14 17:21:55 +01:00
Louis Sautier 650a37b74c Fix typos. 2015-12-14 17:15:18 +01:00
Audrius Butkevicius bb83bd87b9 Merge pull request #92 from devste/patch-1
Master mode: rephrasing how data flows.
2015-12-10 23:14:58 +00:00
Stephen 309829692b Typo 2015-12-10 22:20:39 +01:00
Stephen 053208d3c4 Update bep-v1.rst
I had to read this section about three times, before realizing that the wording did actually describe what's in the picture. Changed the wording slightly to highlight in which direction the data does flow.
2015-12-10 22:16:30 +01:00
Audrius Butkevicius 24d93808ad Merge pull request #91 from 4cdn/patch-1
typo
2015-12-06 15:29:10 +00:00
4cdn 8eec93a6ec typo 2015-12-06 10:23:31 -05:00
Audrius Butkevicius 6b11bc7625 Merge pull request #89 from canton7/feature/relay-docs
Add to the Relays page
2015-12-02 19:29:49 +00:00
Antony Male 77c8906217 Add to the Relays page
- Minimum recommended specifications
 - More detail on how to run a relay on port 443
2015-12-02 17:21:22 +00:00
Stefan Tatschner 6d94f321cd Add syncthing-relay(7) 2015-12-01 15:33:19 +01:00
NickPyz e1f51b2973 Update security.rst 2015-12-01 07:45:19 +01:00
Audrius Butkevicius 8946b20902 Merge pull request #90 from canton7/feature/debug-port
Fix port in example in rest/system-debug-post
2015-11-30 08:34:56 +00:00
Antony Male 7c541140d0 Fix port in example in rest/system-debug-post 2015-11-30 08:20:06 +00:00
Jakob Borg f2c244b761 Draft hash algo doc 2015-11-28 21:44:42 +01:00
Jakob Borg 0aeffe7713 Merge pull request #88 from andersonvom/master
Fix formatting and typo on ignoring.rst
2015-11-26 08:06:46 +01:00
Anderson Mesquita ef0f34b0df Fix formatting and typo on ignoring.rst
This fixes a typo at the end of the ignoring.rst file and takes the
opportunity to improve the formatting of the RST file a bit and add some
missing commas.
2015-11-25 20:25:19 -05:00
Audrius Butkevicius 7d8fa4342b Merge pull request #87 from kc1212/patch-1
Missing lib/protocol in source layout
2015-11-24 23:13:24 +00:00
alex2108 06b3cf4cfb Remove options to externally set username and password (fixes #82) 2015-11-24 17:05:14 +01:00
kc1212 61c0db1ecf Missing lib/protocol in source layout
Not sure whether the description is correct.
2015-11-24 00:03:42 +01:00
Jakob Borg 68d4760f9f Merge pull request #85 from Lockszmith/patch-1
Added Windows Task Scheduler solution.
2015-11-23 19:23:05 +01:00
Gal Szkolnik 8489f7bc88 Update autostart.rst
typos fixed, changed brackets to double quotes
2015-11-23 10:44:57 -05:00
Gal Szkolnik 538aaa8ad3 Added Windows Task Scheduler solution.
This solution does not require 3rd Party tools.
2015-11-23 10:26:00 -05:00
Audrius Butkevicius 801133628c Merge pull request #84 from mess110/patch-1
add SUSE Studio appliance link
2015-11-21 17:22:49 +00:00
Cristian Mircea Messel 864961978c add SUSE Studio appliance link 2015-11-21 19:15:39 +02:00
Jakob Borg 1bdec60772 Better example URLs for upgrade JSON 2015-11-17 08:27:28 +01:00
Audrius Butkevicius ad919c9162 Merge pull request #83 from xor-gate/master
Update user contributed Mac OS X app/dmg package
2015-11-16 21:42:02 +00:00
Jerry Jacobs b928f42827 * OSX app new username 2015-11-16 21:29:11 +01:00
Jerry Jacobs 53d472f1a0 * Update syncthing-mac app/dmg 2015-11-16 21:27:56 +01:00
Jakob Borg b88b1bf76c Note that POST /rest/system/discovery no longer exists in v0.12 2015-11-16 19:43:12 +01:00
Jakob Borg 6e4828ff96 The build slaves are offline for the moment 2015-11-16 12:14:40 +01:00
alex2108 6d5cbaaa5f Usage reort update for v0.12 2015-11-14 17:05:36 +01:00
Jakob Borg b4d3ad8eaa The need call is also expensive 2015-11-13 08:17:46 +01:00
Audrius Butkevicius 04b5f362f4 Merge pull request #77 from syncthing/revert-76-patch-1
Revert 1c5690176b
2015-11-09 08:30:22 -05:00
Audrius Butkevicius da64747ee3 Revert 1c5690176b 2015-11-09 08:30:16 -05:00
Audrius Butkevicius 1c5690176b Merge pull request #76 from mlda065/patch-1
Made "https://localhost:8384/" clickable
2015-11-09 07:37:10 -05:00
Matthew Davis ffa1f28f75 Made "https://localhost:8384/" clickable 2015-11-09 20:03:41 +11:00
Jakob Borg 8c3741412f Merge pull request #75 from alex2108/master
/rest/system/connections update for v0.12
2015-11-08 18:12:42 +01:00
Jakob Borg 1136cbe27e Clarify discovery server ID in URL 2015-11-08 14:38:01 +01:00
alex2108 8c76d69c46 /rest/system/connections update for v0.12 2015-11-07 17:46:32 +01:00
Jakob Borg a7248dfdcd Merge pull request #70 from syncthing/discosrv
Procedure for v0.12 discovery server
2015-11-07 14:47:10 +01:00
Audrius Butkevicius cad2102853 Merge pull request #64 from burkemw3/syncing-configuration-files
Add information about syncing config files safely
2015-11-06 23:31:14 -05:00
Matt Burke ccb488790b Add information about syncing config files safely 2015-11-06 15:49:50 -05:00
Audrius Butkevicius fee49d07cf Merge pull request #74 from syncthing/review
Clarify team membership and review policy
2015-11-06 18:17:39 +00:00
Jakob Borg 5244ac68ad Clarify team membership and review policy 2015-11-06 19:05:10 +01:00
Audrius Butkevicius 2eecee3d67 Merge pull request #73 from Stefan-Code/master
added HOME variable to fix #72
2015-11-05 17:39:46 +00:00
Stefan-Code b201abb0f1 added missing slash 2015-11-05 18:32:54 +01:00
Stefan-Code 353eeddfe1 added HOME variable to fix #72 2015-11-05 18:29:16 +01:00
Audrius Butkevicius 009e78bcca Merge pull request #71 from syncthing/firewall
Firewall update for v0.12
2015-11-04 18:09:16 +00:00
Audrius Butkevicius 1b30be1f8d Merge pull request #68 from syncthing/v12
REST and config file updates for v0.12
2015-11-04 17:35:46 +00:00
Jakob Borg b2f00c5994 Fix compiler error/warnings 2015-11-04 17:33:49 +00:00
Jakob Borg 11d0b1a609 Firewall update for v0.12 2015-11-04 17:28:55 +00:00
Jakob Borg d5d0bd946b Update infrastructure for v0.12 2015-11-04 17:22:21 +00:00
Jakob Borg bdad1afa4c We're now documenting v0.12 2015-11-04 17:13:30 +00:00
Jakob Borg ecb7d0bb51 REST changes around logging 2015-11-04 17:06:11 +00:00
Jakob Borg a40ee17ea3 Update config file docs 2015-11-04 17:06:11 +00:00
Jakob Borg b1395a5350 Procedure for v0.12 discovery server 2015-11-04 17:01:44 +00:00
Audrius Butkevicius 6428ece8c6 Merge pull request #69 from syncthing/relaysrv
Procedure to run a relay server
2015-11-04 16:56:40 +00:00
Jakob Borg 929bebb70c Procedure to run a relay server 2015-11-04 16:44:54 +00:00
Audrius Butkevicius e13aaff239 Merge pull request #67 from AudriusButkevicius/relay
Add relay protocol definition
2015-11-03 18:54:39 +00:00
Audrius Butkevicius dea7fd0504 Add relay protocol definition 2015-11-03 18:53:55 +00:00
Audrius Butkevicius 3324a3f4e6 Merge pull request #65 from vhbit/local-disco-v3-magic
Added a expected value for `Announce.Magic`
2015-10-27 10:25:11 +00:00
Valerii Hiora 0c46aa6ff4 Added a expected value for Announce.Magic 2015-10-27 11:42:25 +02:00
Stefan Tatschner e64564c9ae Add syncthing-globaldisco.7 2015-10-20 14:03:36 +02:00
Jakob Borg a9a26be859 Global discovery specs 2015-10-20 13:50:39 +02:00
Jakob Borg de207e4f92 Update specs for v0.12 2015-10-20 13:35:55 +02:00
Jakob Borg 7c2bcdd069 Merge pull request #63 from burkemw3/fix-firewall-link-on-getting-started
Fix Firewall Setup link on Getting Started page
2015-10-20 06:14:36 +02:00
Matt Burke d2b9c7117f Fix Firewall Setup link on Getting Started page 2015-10-19 17:33:26 -04:00
Stefan Tatschner 5790a0f728 Merge pull request #62 from terrycloth/proofreading
Clarify things which confused me my first time trying to set up autostart
2015-10-17 10:18:09 +02:00
terrycloth 259dd17097 Clarify things which confused me my first time trying to set up autostart.
1. For Ubuntu-like systems, I removed the line about clicking to start the
dashboard because it was unnecessarily specific to Unity. There are plenty of
Ubuntu flavors and derivative distros which use other desktops.

2. For the systemd sections about setting up user and system services, I thought
it would make more sense to explain the difference between user- and
system-level services *before* telling the user how to enable them.

3. Pairs of systemd commands where you would run one or the other depending on
how you set up the service should be labeled as such. Otherwise, inexperienced
readers might think you should run the commands in succession.

4. umask is a tricky subject for people who haven't used it before, so I thought
linking to an article that explains it would be helpful.
2015-10-10 23:16:49 -07:00
Audrius Butkevicius b9c60f6e46 Merge pull request #61 from graboluk/patch-1
added stiko to GUI wrappers
2015-10-10 18:29:56 +01:00
graboluk 0a032f1d90 added stiko to GUI wrappers 2015-10-10 18:19:12 +01:00
Jakob Borg 26232729ec Merge pull request #59 from phil-davis/r1
General doc grammar and typos
2015-10-09 02:19:20 +09:00
Phil Davis 9d40bbe3fd General doc grammar and typos 2015-10-08 22:06:15 +05:45
Stefan Tatschner eb00f9e1de Properly format Permissions section 2015-10-06 17:35:53 +02:00
Stefan Tatschner 20f11b8db0 syncthing-inotify is in the official arch linux repos now 2015-10-01 08:37:55 +02:00
Audrius Butkevicius aeddb98e7b Merge pull request #57 from syncthing/obsolete
Move dead or dying projects to the bottom.
2015-09-23 19:50:48 +01:00
Jakob Borg 2c7e4c7abf Merge pull request #45 from simplypeachy/patch-1
Update ignoring.rst
2015-09-23 20:45:33 +02:00
Jakob Borg 5dd66fd7b3 Move dead or dying projects to the bottom.
These are things not updated in the last six months and which don't look
like they are still viable. I don't want to kill them outright, but I
don't want to point users to them either. Other opinions?
2015-09-23 20:38:52 +02:00
Jakob Borg 1871865f25 Re-add license information 2015-09-23 14:47:35 +02:00
Stefan Tatschner 6470d651bb Revert "Add version information to manpage filename"
This reverts commit 3949222974.

Sorry for the noise. :/
2015-09-23 10:20:39 +02:00
Stefan Tatschner 3949222974 Add version information to manpage filename 2015-09-23 10:17:58 +02:00
Jakob Borg 45bc344838 Merge pull request #55 from syncthing/man-updates
Enable specs manpages
2015-09-23 10:15:00 +02:00
Stefan Tatschner caf42ad394 Enable specs manpages 2015-09-23 09:26:53 +02:00
Audrius Butkevicius 9dc75d7f8f Merge pull request #54 from akissa/master
* Link to new Python REST API Bindings
2015-09-22 23:37:49 +01:00
Andrew Colin Kissa cd659a5a3c * Link to new Python REST API Bindings 2015-09-23 00:24:22 +02:00
Jakob Borg ebb98a13e2 Add BEP & discovery protocols 2015-09-22 23:30:39 +02:00
Jakob Borg afce1b8cd3 Fix minor formatting errors 2015-09-22 22:27:45 +02:00
Jakob Borg 53930687d6 Merge pull request #53 from bigscoop/patch-1
fix typo
2015-09-21 22:04:35 +02:00
bigscoop 8db2533c4c fix typo
Fixed typo
2015-09-21 21:51:02 +02:00
Audrius Butkevicius c2377d0fbf Merge pull request #52 from sieren/patch-2
Update contrib.rst
2015-09-17 21:09:19 +01:00
Matt Sieren d040ba7713 Update contrib.rst
Updated contributions and moved QSyncthingTray to cross-platform section since it now supports Mac, Windows and Linux.
2015-09-17 22:00:06 +02:00
Jakob Borg 99de7e0906 Link to official releases 2015-09-10 14:45:12 +02:00
Jakob Borg 0be393d045 Spelling, prerelease notes 2015-09-10 14:36:14 +02:00
Jakob Borg 015746163f Document custom upgrade server 2015-09-10 14:28:50 +02:00
Stefan Tatschner 9af1eeb559 Rewrap synopsis verbatim part 2015-09-07 21:20:52 +02:00
Stefan Tatschner 134131c04b Merge pull request #48 from cbhushan/patch-1
Added the newly added -logfile option
2015-09-07 21:18:58 +02:00
Stefan Tatschner 44b4bbeef6 Merge pull request #49 from cbhushan/patch-2
Mention -logfile option in FAQ
2015-09-07 21:16:02 +02:00
C Bhushan 99dca43050 updated with -logfile option 2015-09-07 11:39:05 -07:00
C Bhushan 9a16073269 Added the newly added -logfile option 2015-09-07 11:26:46 -07:00
Jakob Borg 8634ef23c1 more speling 2015-09-07 08:39:43 +02:00
Jakob Borg 9077d68d46 spling 2015-09-07 08:37:59 +02:00
Stefan Tatschner 317ce61814 Merge pull request #47 from sieren/patch-1
Update contrib.rst

Added QSyncthingTray to list of OSX Contributions.
2015-09-05 15:01:10 +02:00
Matt Sieren a8a72fe88a Update contrib.rst
Added QSyncthingTray
2015-09-05 14:52:45 +02:00
Jakob Borg 214ad8b9b8 Update contrib.rst 2015-09-04 10:10:04 +02:00
Jakob Borg 7ed969feba Update contrib.rst 2015-09-04 10:08:59 +02:00
Stefan Tatschner 134d64616c Fix underline length 2015-09-03 10:10:58 +02:00
Jakob Borg ac6440ab8b Merge pull request #46 from majedev/patch-1
Fix StartupComplete event documentation title
2015-09-03 07:56:33 +02:00
Majed Abdulaziz 70887ac325 Update startupcomplete.rst
Event title is StartupCompleted, while actual event type is StartupComplete (without d). Should be consistent with event type.
2015-09-03 05:58:09 +03:00
simplypeachy d75c5cae25 Update ignoring.rst 2015-09-02 22:06:33 +01:00
simplypeachy 54d0e2201e Update ignoring.rst
Explicitly specify that #include'd files must be within the repo. Note about Mac/Windows always being case-insensitive.
2015-09-02 20:59:51 +01:00
Jakob Borg f66a9046e7 Discovery clarifications 2015-09-02 17:36:15 +02:00
Audrius Butkevicius f7573e99a7 Merge pull request #44 from simplypeachy/patch-4
Update getting-started.rst
2015-09-02 01:21:08 +01:00
Audrius Butkevicius 23d54affea Merge pull request #43 from simplypeachy/patch-3
Minor grammatical clean-up
2015-09-02 01:20:57 +01:00
simplypeachy dc699980c7 Update getting-started.rst
Minor grammatical improvements, cleaned up some sentences and added a few useful sentences. Mention of the requirement of browser cookies as these seem to trip up some new users, looking at the forum and reddit.
2015-09-02 00:51:57 +01:00
simplypeachy 7dbf077e29 Minor grammatical clean-up 2015-09-02 00:04:15 +01:00
Jakob Borg fad1014d40 Add infrastructure article 2015-09-01 09:44:47 +02:00
Jakob Borg 33a8093e67 Merge pull request #41 from fearedbliss/patch-1
Including additional repository for acquiring Syncthing on Gentoo
2015-09-01 07:20:55 +02:00
Jonathan Vasquez 99987afab7 Including additional repository for acquiring Syncthing on Gentoo 2015-08-31 17:28:52 -04:00
Jakob Borg d82d20201c Describe new release signatures 2015-08-31 21:18:06 +02:00
Jakob Borg 8f74785064 New release process 2015-08-30 20:59:14 +02:00
Jakob Borg 565322be5d Multiple GUI:s are fine nowadays 2015-08-30 20:50:40 +02:00
Jakob Borg d2413c30eb Update issues.rst 2015-08-20 21:26:24 +02:00
Jakob Borg 7e741ca388 Update source directory structure 2015-08-13 12:39:46 +02:00
Jakob Borg 2761f8db9e Merge pull request #37 from raferobinson/patch-1
Update Command Line Operation: Synopsis & Options
2015-08-09 09:06:44 +02:00
raferobinson b203e8143f Update syncthing.rst
Added "On Windows only" -no-console in Option section.
2015-08-08 22:43:06 -04:00
Audrius Butkevicius 4795922c46 Merge pull request #38 from syncthing/Zillode-api-key-curl
Show example with api-key using curl
2015-08-08 20:07:05 +01:00
Zillode 81ef3e2859 Show example with api-key using curl
I always need to look for this, so I think it might be useful here
2015-08-08 21:02:56 +02:00
raferobinson cf96200290 Update syncthing.rst
Added -no-console and description.
2015-08-08 00:22:39 -04:00
Jakob Borg 3f355d26cd Merge pull request #34 from syncthing/Zillode-21026
Document 21026/UDP
2015-08-06 08:23:07 +02:00
Audrius Butkevicius 4fe8af25b7 Merge pull request #36 from simplypeachy/patch-2
Minor typo/grammatical fixes.
2015-08-05 21:49:15 +01:00
simplypeachy 33000e8dea Minor typo/grammatical fixes. 2015-08-05 21:45:00 +01:00
Zillode b171371b2a Document 21026/UDP 2015-07-31 23:40:48 +02:00
Audrius Butkevicius 8d0a22c9ce Merge pull request #33 from cron410/patch-1
Corrected mistype of "have"
2015-07-27 19:38:03 +01:00
cron410 5ed23f7990 Corrected mistype of "have" 2015-07-27 14:27:49 -04:00
Audrius Butkevicius a46c24dd23 Merge pull request #32 from theincogtion/patch-2
Update autostart.rst -> permissions
2015-07-24 23:16:13 +01:00
theincogtion 6a91e70f8f Update autostart.rst 2015-07-25 00:12:56 +02:00
Brian 4ab2357276 Fixed versioning seealso 2015-07-23 23:27:53 +02:00
Jakob Borg 2d886db1a4 Merge pull request #28 from Canisaur/patch-1
Small fixes to config.rst
2015-07-22 15:05:37 +02:00
Brian bd99796c8a Fix bracket description, add versioning seealso
IPv6 will not work with angle <> brackets.  It uses [] square brackets.

Add section label to versioning

Added section label to versioning for cross-ref
2015-07-22 08:44:49 -04:00
Jakob Borg 12770ac455 Document autoNormalize, default value for ignoreDelete 2015-07-21 21:56:30 +02:00
Jakob Borg 5c3e5b8ea8 Merge pull request #29 from syncthing/ignoredelete
Document ignoreDelete
2015-07-21 20:51:25 +02:00
Jakob Borg eefb289e86 Document ignoreDelete 2015-07-21 20:38:23 +02:00
Jakob Borg 2f6e2a585c Add documentation for running a discovery server 2015-07-19 11:49:37 +02:00
Stefan Tatschner f9077d4db0 Merge pull request #27 from kreischweide/patch-1
Removed metrothing windows client
2015-07-13 12:57:37 +02:00
Adrian Rudnik 0e9ce0f409 Removed metrothing windows client
Had to deprecate the solution, no more time to maintain it due to job and other constraints.
2015-07-13 11:19:31 +02:00
Audrius Butkevicius ae80d192ec Merge pull request #26 from canton7/feature/db-file-get
Add query parameter docs to db-file-get
2015-07-11 14:00:53 +01:00
Antony Male 8a2d360e2e Add query parameter docs to db-file-get 2015-07-11 13:17:22 +01:00
Audrius Butkevicius 72cb8817fe Merge pull request #25 from lfam/master
Fix REST discovery API documentation
2015-07-09 09:22:36 +01:00
Leo Famulari 57c3d34c78 Fix REST discovery API documentation
Replace reference to nonexistent "hint" URI for POSTing to the discovery
cache with correct reference:
/rest/system/discovery/hint -> /rest/system/discovery
2015-07-08 22:57:20 -04:00
Jean-Denis Vauguet 1940d5fbfa Fix link to Community Contributions from dev/intro
[Stefan Tatschner]: Added missing link title
2015-07-06 09:44:00 +02:00
Jakob Borg b5c71c2076 draft 2015-07-01 22:16:54 +02:00
Jakob Borg d36508766d Coding style 2015-07-01 17:30:08 +02:00
Jakob Borg f04b3155a3 draft localver 2015-07-01 15:17:06 +02:00
Jakob Borg 19a7e40456 Grammar 2015-07-01 13:31:24 +02:00
Stefan Tatschner 34aed35949 Rewrap users/config.rst; 2015-07-01 08:33:07 +02:00
Stefan Tatschner 612ada6a09 Add literal backticks 2015-07-01 08:32:54 +02:00
Jakob Borg 0612e1cee5 Merge pull request #22 from syncthing/failed-files
FolderErrors event
2015-06-30 19:25:29 +02:00
Jakob Borg 8de7180338 FolderErrors event 2015-06-30 14:44:16 +02:00
Audrius Butkevicius 8c3d3a3b2a Merge pull request #23 from arneko/patch-1
Update config.rst
2015-06-29 17:45:37 +01:00
arneko 6c71c48613 Update config.rst
Added some minimal documentation for ping-related configuration
2015-06-29 14:26:38 +02:00
Jakob Borg d980bdbbfc Update release.rst 2015-06-28 08:12:29 +02:00
Audrius Butkevicius d26a92e494 Update syncthing.rst 2015-06-26 08:43:12 +01:00
Jakob Borg 29670c85ce Merge pull request #20 from syncthing/AudriusButkevicius-patch-1
Update faq.rst
2015-06-23 20:50:16 +02:00
Audrius Butkevicius bd35b7f006 Merge pull request #21 from simplypeachy/patch-1
Removed minor grammatical error
2015-06-23 11:17:01 +01:00
simplypeachy 9778ffb967 Removed minor grammatical error 2015-06-23 10:54:22 +01:00
Audrius Butkevicius 2214201990 Update faq.rst 2015-06-22 17:00:20 +01:00
Jakob Borg 396524252c Local sessions 2015-06-21 23:35:12 +02:00
Jakob Borg c07414cda5 Local sessions 2015-06-21 23:34:48 +02:00
Jakob Borg 70734b42a1 Merge pull request #11 from syncthing/Zillode-cookies
Explain CSRF tokens & cookies
2015-06-21 23:29:22 +02:00
Jakob Borg dc7c973c42 Followsymlinks tweaks 2015-06-21 23:28:38 +02:00
Jakob Borg 0b00ef65c2 Draft docs for FollowSymlinks 2015-06-20 22:56:37 +02:00
Zillode 6aeb42602f Clarify session 2015-06-20 13:43:30 +02:00
Jakob Borg f0d6e9aeba Improve formatting of build commands 2015-06-15 20:52:38 +02:00
Jakob Borg 8cada19fd3 Minor tweaks... 2015-06-15 00:16:48 +02:00
Jakob Borg 22f3e392ad Link syntax 2015-06-15 00:06:19 +02:00
Jakob Borg 8b2d790131 Update configuration description 2015-06-14 23:55:40 +02:00
Jakob Borg d07c5b07b1 ItemFinished/ItemStarted metadata 2015-06-14 23:03:02 +02:00
Jakob Borg 2ffe2b7226 Introduction to file versioning (fixes #17) 2015-06-14 13:48:39 +02:00
Jakob Borg 76a8ef14f8 Correct contribution guidelines link 2015-06-13 22:40:09 +02:00
Jakob Borg ec9ade33c2 Update /rest/system/reset to new behavior 2015-06-13 22:34:37 +02:00
Jakob Borg 000536ca76 Describe trash can versioning 2015-06-12 16:29:04 +02:00
Jakob Borg 86647d6ebb Merge pull request #16 from AudriusButkevicius/AudriusButkevicius-patch-1
Update selective.rst
2015-06-11 11:27:24 +02:00
Audrius Butkevicius eaa1663d9d Update selective.rst 2015-06-11 10:13:21 +01:00
Jakob Borg dd74dffe3a selective sync draft 2015-06-11 09:15:53 +02:00
Jakob Borg 61ae7c3100 selective sync draft 2015-06-11 08:19:03 +02:00
Jakob Borg d707643ce6 Fix lists in gui intro 2015-06-10 17:43:07 +02:00
Jakob Borg 9f4325980f Add draft selective sync doc 2015-06-10 17:39:47 +02:00
Jakob Borg 2a09fa5ce4 Reflow it back. :) 2015-06-10 14:01:52 +02:00
Jakob Borg bb04cbbf8b Reformat faq entry 2015-06-10 13:59:40 +02:00
Stefan Tatschner e59772d3df Definition lists everywhere
I just found out that they can be nested, which is useful in this case.
2015-06-10 01:01:59 +02:00
Stefan Tatschner c62fd6af98 Use .. cmdoption:: to document the cli 2015-06-10 00:59:58 +02:00
Jakob Borg 716a75645e Add contribution guidelines 2015-06-10 00:00:12 +02:00
Jakob Borg 605e8d04c6 Remove spurious space in device ID 2015-06-09 23:38:39 +02:00
Jakob Borg 0d946a3401 Clean up the dev intro a little, add source code tree 2015-06-09 23:36:14 +02:00
Jakob Borg 10834febdd Merge pull request #14 from rumpelsepp/apache-24
reverseproxy: Apache 2.2 is rather old, switch to 2.4
2015-06-07 00:10:44 +02:00
Stefan Tatschner fd865c3d14 device-ids: Use proper code blocks 2015-06-06 22:43:07 +02:00
Stefan Tatschner fc70cf4e6a issues.rst: Use definition lists instead of plain lists 2015-06-06 14:06:03 +02:00
Stefan Tatschner 6e14b63175 reverseproxy: Fix obsolete port 2015-06-06 12:01:47 +02:00
Stefan Tatschner 02a9bf7a19 reverseproxy: Apache 2.2 is rather old, switch to 2.4
Apache 2.2 is rather old, IIRC almost debian now ships with 2.4. So
let's document the most recent one.
2015-06-06 11:56:03 +02:00
Stefan Tatschner ba04d232e8 Fix typos and formatting in reverseproxy.rst
It is:

  .. code-block:: LANGUAGE

      Awesome Sourcecode
2015-06-06 11:51:28 +02:00
Jakob Borg 9f2b003c76 Update reverseproxy.rst 2015-06-06 07:18:58 +02:00
Jakob Borg 108b645079 Update reverseproxy.rst 2015-06-06 07:17:18 +02:00
Jakob Borg 8c4454ee64 Update reverseproxy.rst 2015-06-06 07:15:33 +02:00
Jakob Borg 5ec61b3712 Merge pull request #13 from zukoo/patch-1
Create reverseproxy.rst
2015-06-06 07:12:08 +02:00
François-Xavier Gsell ccc9f644ef Create reverseproxy.rst 2015-06-06 10:24:01 +08:00
Jakob Borg 76a1876d10 Merge pull request #10 from vikbez/patch-1
Added an osx 10.9 compatible client link
2015-06-04 19:29:01 +02:00
Zillode a3c3203ab1 Explain CSRF tokens & cookies 2015-06-04 15:08:26 +02:00
Romain Gay 2445726123 Update contrib.rst
added osx 10.9 client link
2015-06-04 14:43:25 +02:00
Jakob Borg 2c31f04547 Clean away .ropeproject 2015-06-03 19:46:27 +02:00
Jakob Borg f9fda9eca4 Fix blockquote and li font sizes 2015-06-03 19:46:27 +02:00
Jakob Borg 08e652b2a2 Add development content from old wiki 2015-06-03 19:46:27 +02:00
Jakob Borg 7bb9daec42 Merge pull request #9 from timboudreau/master
Add Gentoo Linux instructions
2015-06-01 10:25:55 +02:00
Tim Boudreau 3aeb01235e Add Gentoo Linux instructions 2015-06-01 04:12:56 -04:00
Jakob Borg e62be987f9 Generate selected PDF:s 2015-06-01 09:14:27 +02:00
Jakob Borg 033286ab6f Figures are in fact images 2015-06-01 09:04:14 +02:00
Jakob Borg 70d0b9d201 Rudimentary docs for ext file versioning 2015-06-01 08:50:06 +02:00
Jakob Borg 914fef83e9 Slightly clarify versioning 2015-06-01 08:29:59 +02:00
Stefan Tatschner 4f2b1e5d04 Add a TODO, since the debugging page duplicates syncthing(1) 2015-05-31 10:31:07 +02:00
Stefan Tatschner a0aef6e94d Add :user:`` links 2015-05-31 10:26:49 +02:00
Stefan Tatschner 0b4ee46f13 Remove Syncthing prefix from syncthing-config(5) in html toc 2015-05-31 10:15:50 +02:00
Jakob Borg fe9341dc28 Title case See Also 2015-05-31 10:13:01 +02:00
Jakob Borg 4ae12a57c0 Fix toc title for man page 2015-05-31 10:09:25 +02:00
Jakob Borg e2467731a3 Simplify footer 2015-05-31 10:06:57 +02:00
Jakob Borg c74d212da0 De-retinize images 2015-05-31 10:03:09 +02:00
Jakob Borg 1aeb5516d7 Build pdf 2015-05-30 11:29:21 +02:00
Jakob Borg 425d2ed163 Build man pages 2015-05-30 10:41:13 +02:00
Jakob Borg 84e2295834 Typos 2015-05-30 08:31:21 +02:00
Jakob Borg e29443e5dc Update building.rst 2015-05-30 08:29:22 +02:00
Jakob Borg 665ce7aa58 Note about GOPATH 2015-05-30 08:24:24 +02:00
Stefan Tatschner debef5cf6e Merge exit codes page into syncthing(1) 2015-05-30 00:30:01 +02:00
Stefan Tatschner 428c3638c1 Enable syncthing(1) page 2015-05-30 00:15:11 +02:00
Stefan Tatschner 6ef06bb854 Create syncthing(1) 2015-05-30 00:15:02 +02:00
Stefan Tatschner 5d5d5394a0 Fix broken .. seealso:: 2015-05-29 23:34:41 +02:00
Stefan Tatschner 348cd30fc2 Reformat project-presentation.rst and use external links ext 2015-05-29 23:34:41 +02:00
Stefan Tatschner a9afba6c21 Wrap text and use proper rst links on index.rst 2015-05-29 23:34:41 +02:00
Stefan Tatschner 385c46fd4b Use sphinx.ext.extlinks to generate frequently used links
Documentation see here:
http://sphinx-doc.org/ext/extlinks.html
2015-05-29 23:08:17 +02:00
Stefan Tatschner 6ad32bfe1b Add a favicon
Although there is a warning about not being an *.ico, I think the
warning can be safely ignored. Modern browsers support pngs well.
2015-05-29 22:55:47 +02:00
Stefan Tatschner 0c2d769dee Update the README
Since we are using rst now, let's convert the README to rst as well. I
also removed some obsolete Jekyll instructions and replaced them with
the proper Sphinx ones.
2015-05-29 22:48:26 +02:00
Jakob Borg 5f7f96bed3 Fix *most* of monospaced font sizes
grep -lr 'font-size: 12px' bower_components/wyrm | xargs sed -i "" 's/font-size: 12px/font-size: 15px/'

There's still a font-size: 75% in there that I haven't got to.
2015-05-29 18:02:43 +02:00
Jakob Borg 31a396bc86 Merge pull request #7 from rumpelsepp/man
Create and enable manpages
2015-05-29 17:04:26 +02:00
Jakob Borg 5b37d1b497 Add patched sphinx_rtd_theme
There must be a cleaner way to do this... Currently, this theme is built
with:

--- a/sass/_theme_layout.sass
+++ b/sass/_theme_layout.sass
@@ -292,7 +292,7 @@
 .wy-nav-content
   padding: $gutter $gutter * 2
   height: 100%
-  max-width: 800px
+  max-width: 1000px
   margin: auto

 .wy-body-mask
@@ -354,13 +354,6 @@ footer
       height: 100%
       overflow: hidden

-+media($desktop-wider)
-  .wy-nav-content-wrap
-    background: rgba(0,0,0,.05)
-  .wy-nav-content
-    margin: 0
-    background: $section-background-color
-
 @media print
   .rst-versions, footer, .wy-nav-side
     display: none

and this on bower_components/wyrm/sass/wyrm_core/_wy_variables.sass

- $base-font-size:                      16px !default
+ $base-font-size:                      18px !default
  $base-font-family:                    "Helvetica Neue", Arial, sans-serif !default
  $base-font-family-bold:               bold
- $base-line-height:                    24px !default
+ $base-line-height:                    27px !default
2015-05-29 16:58:12 +02:00
Stefan Tatschner ef76180205 Enable the manpages in the configuration file
This patch enables the previously created manpages in the sphinx
configuration file. It also enables "man_show_urls" because the
manpages include several external links.
2015-05-29 16:45:03 +02:00
Stefan Tatschner b7c37f330d Create syncthing-networking(7) 2015-05-29 16:45:03 +02:00
Stefan Tatschner 0e884a9b65 Create syncthing-security(7) 2015-05-29 16:45:03 +02:00
Stefan Tatschner dada1daded Create syncthing-versioning(7) 2015-05-29 16:45:03 +02:00
Stefan Tatschner e6d0b370d2 Create syncthing-faq(7) 2015-05-29 16:44:16 +02:00
Stefan Tatschner a979a50124 Create syncthing-rest-api(7) 2015-05-29 16:43:13 +02:00
Stefan Tatschner bcee82cdde Create syncthing-device-ids(7) 2015-05-29 16:42:28 +02:00
Jakob Borg 3d76603ece Merge pull request #5 from rumpelsepp/cleanups
Some cleanups
2015-05-29 16:39:28 +02:00
Stefan Tatschner 0215ab696a Create syncthing-event-api(7) 2015-05-29 16:39:22 +02:00
Stefan Tatschner 855a77234f Create syncthing-stignore(5) 2015-05-29 16:39:22 +02:00
Stefan Tatschner 0d06e47acf Create syncthing-config(5)
Convert users/config.rst to a properly formatted manpage
2015-05-29 16:39:22 +02:00
Jakob Borg 2e407d3cf7 Merge pull request #3 from rumpelsepp/master
Some rst fixes
2015-05-29 16:37:44 +02:00
Jakob Borg ff64e58b41 Merge pull request #8 from rumpelsepp/todos
Enable TODO extension
2015-05-29 16:34:08 +02:00
Stefan Tatschner dd410c92f1 Fix some rst styling things in autostart.rst 2015-05-29 16:26:09 +02:00
Stefan Tatschner e3cb19f311 Do not count by hand
Humans are too stupid to count, so let the computer do the actual work!
2015-05-29 16:24:58 +02:00
Stefan Tatschner e5242d68b4 Change .. code:: to .. code-block::
Sphinx uses .. code-block:: directives to enable highlighting:
http://sphinx-doc.org/markup/code.html#directive-code-block

Use

  for i in *.rst; do
        sed -i 's/code::/code-block::/g' $i
    done

to update these strings to enable proper highlighting.

Note: A have disabled the windows "bash" one, because the highlighting
is broken anyway...
2015-05-29 16:24:58 +02:00
Jakob Borg 892ad9de74 Merge pull request #2 from alexwlchan/master
A handful of spelling fixes
2015-05-29 16:19:50 +02:00
Stefan Tatschner 89dc32dca0 Enable TODO extension
This extension enables the usage of .. todo:: blocks. This is quite
useful because we can encourage people to work on particular sections
of the documentation.
2015-05-29 14:27:10 +02:00
Jakob Borg 1c45b78ecb Update Python bindings 2015-05-29 13:18:29 +02:00
Stefan Tatschner 635b3f6db5 Remove the font size adjustments
Increasing the font size is *not* useful due to a couple of reasons:

  - It breaks the visual seperation between the headlines. It is very
    hard to determine the headline levels when the font size is that big.

  - It breaks the definition lists which are useful for manpages.

  - It extends the line width to an unnatural value. It is not narrow
    any more, but it is also not big. It just looks a bit odd.

  - The regular font (on linux) renders as almost a light bold type. It
    generates a "Comic Sans" experience. That does not look
    professional.

Since I am not a designer and I don't know how to fix these problems in
the right way, I think the best solution is switching back to the very
sane defaults.
2015-05-29 12:59:37 +02:00
Stefan Tatschner f59300023e Cleanup conf.py
- Move the configuration values to the appropriate position in the file.
- Setting "html_theme_path" is not needed.
- Enable "html_last_updated_fmt". It is quite useful to determine the
  last documentation update. IMO it does not harm at all.
2015-05-29 12:51:35 +02:00
Alex Chan a529eb7fd4 Fix a capitalisation of Syncthing 2015-05-29 09:40:48 +01:00
Alex Chan f6b264275b Spelling fixes 2015-05-29 09:40:00 +01:00
Jakob Borg 5aca789fea Update faq.rst 2015-05-29 09:43:11 +02:00
Jakob Borg afe9f4a845 Update faq.rst 2015-05-29 09:42:47 +02:00
Jakob Borg 8da54bd187 Strikeout 2015-05-29 09:40:38 +02:00
Jakob Borg eb3ec76215 Update project-presentation.rst 2015-05-29 09:29:50 +02:00
Jakob Borg 567d3dd0f5 Sync with latest Wiki content, syncthing -> Syncthing 2015-05-29 09:10:37 +02:00
Jakob Borg cbf0766e86 Formatting fixes 2015-05-28 15:11:12 +02:00
Jakob Borg 040a7f1cf4 Spacing, deploy 2015-05-28 14:19:05 +02:00
Jakob Borg 997d586c5e Style & cleanups 2015-05-28 14:06:57 +02:00
Jakob Borg d0c6272a90 rm -rf 2015-05-28 12:43:56 +02:00
Jakob Borg fd7fe795ad Initial 2015-05-28 12:40:43 +02:00
372 changed files with 14890 additions and 32562 deletions
+3 -5
View File
@@ -1,5 +1,3 @@
syncthing
syncthing.exe
*.tar.gz
*.zip
*.asc
_build/
_deployed
_deployed.old
-22
View File
@@ -1,22 +0,0 @@
Please do contribute!
## Building
[See the wiki](https://github.com/calmh/syncthing/wiki/Building)
## Tests
Yes please!
## Style
`go fmt`
## Documentation
[Hack it here](https://github.com/calmh/syncthing/wiki)
## License
MIT
-37
View File
@@ -1,37 +0,0 @@
{
"ImportPath": "github.com/calmh/syncthing",
"GoVersion": "devel +3ca54dd30864 Sat Mar 22 11:05:40 2014 -0700",
"Packages": [
"./cmd/syncthing"
],
"Deps": [
{
"ImportPath": "code.google.com/p/go.net/ipv6",
"Comment": "null-117",
"Rev": "c17ad62118ea511e1051721b429779fa40bddc74"
},
{
"ImportPath": "code.google.com/p/go.text/transform",
"Comment": "null-81",
"Rev": "9cbe983aed9b0dfc73954433fead5e00866342ac"
},
{
"ImportPath": "code.google.com/p/go.text/unicode/norm",
"Comment": "null-81",
"Rev": "9cbe983aed9b0dfc73954433fead5e00866342ac"
},
{
"ImportPath": "github.com/calmh/ini",
"Rev": "386c4240a9684d91d9ec4d93651909b49c7269e1"
},
{
"ImportPath": "github.com/codegangsta/inject",
"Rev": "9aea7a2fa5b79ef7fc00f63a575e72df33b4e886"
},
{
"ImportPath": "github.com/codegangsta/martini",
"Comment": "v0.1-142-g8659df7",
"Rev": "8659df7a51aebe6c6120268cd5a8b4c34fa8441a"
}
]
}
Generated
-5
View File
@@ -1,5 +0,0 @@
This directory tree is generated automatically by godep.
Please do not edit.
See https://github.com/tools/godep for more information.
-2
View File
@@ -1,2 +0,0 @@
/pkg
/bin
-83
View File
@@ -1,83 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"errors"
"fmt"
"net"
"sync"
)
var (
errMissingAddress = errors.New("missing address")
errInvalidConnType = errors.New("invalid conn type")
errNoSuchInterface = errors.New("no such interface")
)
// References:
//
// RFC 2292 Advanced Sockets API for IPv6
// http://tools.ietf.org/html/rfc2292
// RFC 2460 Internet Protocol, Version 6 (IPv6) Specification
// http://tools.ietf.org/html/rfc2460
// RFC 3493 Basic Socket Interface Extensions for IPv6
// http://tools.ietf.org/html/rfc3493.html
// RFC 3542 Advanced Sockets Application Program Interface (API) for IPv6
// http://tools.ietf.org/html/rfc3542
//
// Note that RFC 3542 obsoltes RFC 2292 but OS X Snow Leopard and the
// former still support RFC 2292 only. Please be aware that almost
// all protocol implementations prohibit using a combination of RFC
// 2292 and RFC 3542 for some practical reasons.
type rawOpt struct {
sync.Mutex
cflags ControlFlags
}
func (c *rawOpt) set(f ControlFlags) { c.cflags |= f }
func (c *rawOpt) clear(f ControlFlags) { c.cflags &^= f }
func (c *rawOpt) isset(f ControlFlags) bool { return c.cflags&f != 0 }
// A ControlFlags reprensents per packet basis IP-level socket option
// control flags.
type ControlFlags uint
const (
FlagTrafficClass ControlFlags = 1 << iota // pass the traffic class on the received packet
FlagHopLimit // pass the hop limit on the received packet
FlagSrc // pass the source address on the received packet
FlagDst // pass the destination address on the received packet
FlagInterface // pass the interface index on the received packet
FlagPathMTU // pass the path MTU on the received packet path
)
// A ControlMessage represents per packet basis IP-level socket
// options.
type ControlMessage struct {
// Receiving socket options: SetControlMessage allows to
// receive the options from the protocol stack using ReadFrom
// method of PacketConn.
//
// Specifying socket options: ControlMessage for WriteTo
// method of PacketConn allows to send the options to the
// protocol stack.
//
TrafficClass int // traffic class, must be 1 <= value <= 255 when specifying
HopLimit int // hop limit, must be 1 <= value <= 255 when specifying
Src net.IP // source address, specifying only
Dst net.IP // destination address, receiving only
IfIndex int // interface index, must be 1 <= value when specifying
NextHop net.IP // next hop address, specifying only
MTU int // path MTU, receiving only
}
func (cm *ControlMessage) String() string {
if cm == nil {
return "<nil>"
}
return fmt.Sprintf("tclass: %#x, hoplim: %v, src: %v, dst: %v, ifindex: %v, nexthop: %v, mtu: %v", cm.TrafficClass, cm.HopLimit, cm.Src, cm.Dst, cm.IfIndex, cm.NextHop, cm.MTU)
}
@@ -1,151 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin
package ipv6
import (
"net"
"os"
"syscall"
"unsafe"
)
const pktinfo = FlagDst | FlagInterface
func setControlMessage(fd int, opt *rawOpt, cf ControlFlags, on bool) error {
opt.Lock()
defer opt.Unlock()
if cf&FlagHopLimit != 0 {
if err := setIPv6ReceiveHopLimit(fd, on); err != nil {
return err
}
if on {
opt.set(FlagHopLimit)
} else {
opt.clear(FlagHopLimit)
}
}
if cf&pktinfo != 0 {
if err := setIPv6ReceivePacketInfo(fd, on); err != nil {
return err
}
if on {
opt.set(cf & pktinfo)
} else {
opt.clear(cf & pktinfo)
}
}
return nil
}
func newControlMessage(opt *rawOpt) (oob []byte) {
opt.Lock()
defer opt.Unlock()
l, off := 0, 0
if opt.isset(FlagHopLimit) {
l += syscall.CmsgSpace(4)
}
if opt.isset(pktinfo) {
l += syscall.CmsgSpace(sysSizeofPacketInfo)
}
if l > 0 {
oob = make([]byte, l)
if opt.isset(FlagHopLimit) {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockopt2292HopLimit
m.SetLen(syscall.CmsgLen(4))
off += syscall.CmsgSpace(4)
}
if opt.isset(pktinfo) {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockopt2292PacketInfo
m.SetLen(syscall.CmsgLen(sysSizeofPacketInfo))
off += syscall.CmsgSpace(sysSizeofPacketInfo)
}
}
return
}
func parseControlMessage(b []byte) (*ControlMessage, error) {
if len(b) == 0 {
return nil, nil
}
cmsgs, err := syscall.ParseSocketControlMessage(b)
if err != nil {
return nil, os.NewSyscallError("parse socket control message", err)
}
cm := &ControlMessage{}
for _, m := range cmsgs {
if m.Header.Level != ianaProtocolIPv6 {
continue
}
switch m.Header.Type {
case sysSockopt2292HopLimit:
cm.HopLimit = int(*(*byte)(unsafe.Pointer(&m.Data[:1][0])))
case sysSockopt2292PacketInfo:
pi := (*sysPacketInfo)(unsafe.Pointer(&m.Data[0]))
cm.IfIndex = int(pi.IfIndex)
cm.Dst = pi.IP[:]
}
}
return cm, nil
}
func marshalControlMessage(cm *ControlMessage) (oob []byte) {
if cm == nil {
return
}
l, off := 0, 0
if cm.HopLimit > 0 {
l += syscall.CmsgSpace(4)
}
pion := false
if cm.Src.To4() == nil && cm.Src.To16() != nil || cm.IfIndex != 0 {
pion = true
l += syscall.CmsgSpace(sysSizeofPacketInfo)
}
if len(cm.NextHop) == net.IPv6len {
l += syscall.CmsgSpace(syscall.SizeofSockaddrInet6)
}
if l > 0 {
oob = make([]byte, l)
if cm.HopLimit > 0 {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockopt2292HopLimit
m.SetLen(syscall.CmsgLen(4))
data := oob[off+syscall.CmsgLen(0):]
*(*byte)(unsafe.Pointer(&data[:1][0])) = byte(cm.HopLimit)
off += syscall.CmsgSpace(4)
}
if pion {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockopt2292PacketInfo
m.SetLen(syscall.CmsgLen(sysSizeofPacketInfo))
pi := (*sysPacketInfo)(unsafe.Pointer(&oob[off+syscall.CmsgLen(0)]))
if ip := cm.Src.To16(); ip != nil && ip.To4() == nil {
copy(pi.IP[:], ip)
}
if cm.IfIndex != 0 {
pi.IfIndex = uint32(cm.IfIndex)
}
off += syscall.CmsgSpace(sysSizeofPacketInfo)
}
if len(cm.NextHop) == net.IPv6len {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockopt2292NextHop
m.SetLen(syscall.CmsgLen(syscall.SizeofSockaddrInet6))
sa := (*syscall.RawSockaddrInet6)(unsafe.Pointer(&oob[off+syscall.CmsgLen(0)]))
setSockaddr(sa, cm.NextHop, cm.IfIndex)
off += syscall.CmsgSpace(syscall.SizeofSockaddrInet6)
}
}
return
}
@@ -1,27 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build dragonfly plan9 solaris
package ipv6
func setControlMessage(fd int, opt *rawOpt, cf ControlFlags, on bool) error {
// TODO(mikio): Implement this
return errOpNoSupport
}
func newControlMessage(opt *rawOpt) (oob []byte) {
// TODO(mikio): Implement this
return nil
}
func parseControlMessage(b []byte) (*ControlMessage, error) {
// TODO(mikio): Implement this
return nil, errOpNoSupport
}
func marshalControlMessage(cm *ControlMessage) (oob []byte) {
// TODO(mikio): Implement this
return nil
}
@@ -1,210 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build freebsd linux netbsd openbsd
package ipv6
import (
"net"
"os"
"syscall"
"unsafe"
)
const pktinfo = FlagDst | FlagInterface
func setControlMessage(fd int, opt *rawOpt, cf ControlFlags, on bool) error {
opt.Lock()
defer opt.Unlock()
if cf&FlagTrafficClass != 0 {
if err := setIPv6ReceiveTrafficClass(fd, on); err != nil {
return err
}
if on {
opt.set(FlagTrafficClass)
} else {
opt.clear(FlagTrafficClass)
}
}
if cf&FlagHopLimit != 0 {
if err := setIPv6ReceiveHopLimit(fd, on); err != nil {
return err
}
if on {
opt.set(FlagHopLimit)
} else {
opt.clear(FlagHopLimit)
}
}
if cf&pktinfo != 0 {
if err := setIPv6ReceivePacketInfo(fd, on); err != nil {
return err
}
if on {
opt.set(cf & pktinfo)
} else {
opt.clear(cf & pktinfo)
}
}
if cf&FlagPathMTU != 0 {
if err := setIPv6ReceivePathMTU(fd, on); err != nil {
return err
}
if on {
opt.set(FlagPathMTU)
} else {
opt.clear(FlagPathMTU)
}
}
return nil
}
func newControlMessage(opt *rawOpt) (oob []byte) {
opt.Lock()
defer opt.Unlock()
l, off := 0, 0
if opt.isset(FlagTrafficClass) {
l += syscall.CmsgSpace(4)
}
if opt.isset(FlagHopLimit) {
l += syscall.CmsgSpace(4)
}
if opt.isset(pktinfo) {
l += syscall.CmsgSpace(sysSizeofPacketInfo)
}
if opt.isset(FlagPathMTU) {
l += syscall.CmsgSpace(sysSizeofMTUInfo)
}
if l > 0 {
oob = make([]byte, l)
if opt.isset(FlagTrafficClass) {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockoptReceiveTrafficClass
m.SetLen(syscall.CmsgLen(4))
off += syscall.CmsgSpace(4)
}
if opt.isset(FlagHopLimit) {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockoptReceiveHopLimit
m.SetLen(syscall.CmsgLen(4))
off += syscall.CmsgSpace(4)
}
if opt.isset(pktinfo) {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockoptReceivePacketInfo
m.SetLen(syscall.CmsgLen(sysSizeofPacketInfo))
off += syscall.CmsgSpace(sysSizeofPacketInfo)
}
if opt.isset(FlagPathMTU) {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockoptReceivePathMTU
m.SetLen(syscall.CmsgLen(sysSizeofMTUInfo))
off += syscall.CmsgSpace(sysSizeofMTUInfo)
}
}
return
}
func parseControlMessage(b []byte) (*ControlMessage, error) {
if len(b) == 0 {
return nil, nil
}
cmsgs, err := syscall.ParseSocketControlMessage(b)
if err != nil {
return nil, os.NewSyscallError("parse socket control message", err)
}
cm := &ControlMessage{}
for _, m := range cmsgs {
if m.Header.Level != ianaProtocolIPv6 {
continue
}
switch m.Header.Type {
case sysSockoptTrafficClass:
cm.TrafficClass = int(*(*byte)(unsafe.Pointer(&m.Data[:1][0])))
case sysSockoptHopLimit:
cm.HopLimit = int(*(*byte)(unsafe.Pointer(&m.Data[:1][0])))
case sysSockoptPacketInfo:
pi := (*sysPacketInfo)(unsafe.Pointer(&m.Data[0]))
cm.Dst = pi.IP[:]
cm.IfIndex = int(pi.IfIndex)
case sysSockoptPathMTU:
mi := (*sysMTUInfo)(unsafe.Pointer(&m.Data[0]))
cm.Dst = mi.Addr.Addr[:]
cm.IfIndex = int(mi.Addr.Scope_id)
cm.MTU = int(mi.MTU)
}
}
return cm, nil
}
func marshalControlMessage(cm *ControlMessage) (oob []byte) {
if cm == nil {
return
}
l, off := 0, 0
if cm.TrafficClass > 0 {
l += syscall.CmsgSpace(4)
}
if cm.HopLimit > 0 {
l += syscall.CmsgSpace(4)
}
pion := false
if cm.Src.To4() == nil && cm.Src.To16() != nil || cm.IfIndex != 0 {
pion = true
l += syscall.CmsgSpace(sysSizeofPacketInfo)
}
if len(cm.NextHop) == net.IPv6len {
l += syscall.CmsgSpace(syscall.SizeofSockaddrInet6)
}
if l > 0 {
oob = make([]byte, l)
if cm.TrafficClass > 0 {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockoptTrafficClass
m.SetLen(syscall.CmsgLen(4))
data := oob[off+syscall.CmsgLen(0):]
*(*byte)(unsafe.Pointer(&data[:1][0])) = byte(cm.TrafficClass)
off += syscall.CmsgSpace(4)
}
if cm.HopLimit > 0 {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockoptHopLimit
m.SetLen(syscall.CmsgLen(4))
data := oob[off+syscall.CmsgLen(0):]
*(*byte)(unsafe.Pointer(&data[:1][0])) = byte(cm.HopLimit)
off += syscall.CmsgSpace(4)
}
if pion {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockoptPacketInfo
m.SetLen(syscall.CmsgLen(sysSizeofPacketInfo))
pi := (*sysPacketInfo)(unsafe.Pointer(&oob[off+syscall.CmsgLen(0)]))
if ip := cm.Src.To16(); ip != nil && ip.To4() == nil {
copy(pi.IP[:], ip)
}
if cm.IfIndex != 0 {
pi.IfIndex = uint32(cm.IfIndex)
}
off += syscall.CmsgSpace(sysSizeofPacketInfo)
}
if len(cm.NextHop) == net.IPv6len {
m := (*syscall.Cmsghdr)(unsafe.Pointer(&oob[off]))
m.Level = ianaProtocolIPv6
m.Type = sysSockoptNextHop
m.SetLen(syscall.CmsgLen(syscall.SizeofSockaddrInet6))
sa := (*syscall.RawSockaddrInet6)(unsafe.Pointer(&oob[off+syscall.CmsgLen(0)]))
setSockaddr(sa, cm.NextHop, cm.IfIndex)
off += syscall.CmsgSpace(syscall.SizeofSockaddrInet6)
}
}
return
}
@@ -1,27 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import "syscall"
func setControlMessage(fd syscall.Handle, opt *rawOpt, cf ControlFlags, on bool) error {
// TODO(mikio): Implement this
return syscall.EWINDOWS
}
func newControlMessage(opt *rawOpt) (oob []byte) {
// TODO(mikio): Implement this
return nil
}
func parseControlMessage(b []byte) (*ControlMessage, error) {
// TODO(mikio): Implement this
return nil, syscall.EWINDOWS
}
func marshalControlMessage(cm *ControlMessage) (oob []byte) {
// TODO(mikio): Implement this
return nil
}
-178
View File
@@ -1,178 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin freebsd linux netbsd openbsd windows
package ipv6
import (
"net"
"syscall"
)
// MulticastHopLimit returns the hop limit field value for outgoing
// multicast packets.
func (c *dgramOpt) MulticastHopLimit() (int, error) {
if !c.ok() {
return 0, syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return 0, err
}
return ipv6MulticastHopLimit(fd)
}
// SetMulticastHopLimit sets the hop limit field value for future
// outgoing multicast packets.
func (c *dgramOpt) SetMulticastHopLimit(hoplim int) error {
if !c.ok() {
return syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return err
}
return setIPv6MulticastHopLimit(fd, hoplim)
}
// MulticastInterface returns the default interface for multicast
// packet transmissions.
func (c *dgramOpt) MulticastInterface() (*net.Interface, error) {
if !c.ok() {
return nil, syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return nil, err
}
return ipv6MulticastInterface(fd)
}
// SetMulticastInterface sets the default interface for future
// multicast packet transmissions.
func (c *dgramOpt) SetMulticastInterface(ifi *net.Interface) error {
if !c.ok() {
return syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return err
}
return setIPv6MulticastInterface(fd, ifi)
}
// MulticastLoopback reports whether transmitted multicast packets
// should be copied and send back to the originator.
func (c *dgramOpt) MulticastLoopback() (bool, error) {
if !c.ok() {
return false, syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return false, err
}
return ipv6MulticastLoopback(fd)
}
// SetMulticastLoopback sets whether transmitted multicast packets
// should be copied and send back to the originator.
func (c *dgramOpt) SetMulticastLoopback(on bool) error {
if !c.ok() {
return syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return err
}
return setIPv6MulticastLoopback(fd, on)
}
// JoinGroup joins the group address group on the interface ifi.
// It uses the system assigned multicast interface when ifi is nil,
// although this is not recommended because the assignment depends on
// platforms and sometimes it might require routing configuration.
func (c *dgramOpt) JoinGroup(ifi *net.Interface, group net.Addr) error {
if !c.ok() {
return syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return err
}
grp := netAddrToIP16(group)
if grp == nil {
return errMissingAddress
}
return joinIPv6Group(fd, ifi, grp)
}
// LeaveGroup leaves the group address group on the interface ifi.
func (c *dgramOpt) LeaveGroup(ifi *net.Interface, group net.Addr) error {
if !c.ok() {
return syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return err
}
grp := netAddrToIP16(group)
if grp == nil {
return errMissingAddress
}
return leaveIPv6Group(fd, ifi, grp)
}
// Checksum reports whether the kernel will compute, store or verify a
// checksum for both incoming and outgoing packets. If on is true, it
// returns an offset in bytes into the data of where the checksum
// field is located.
func (c *dgramOpt) Checksum() (on bool, offset int, err error) {
if !c.ok() {
return false, 0, syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return false, 0, err
}
return ipv6Checksum(fd)
}
// SetChecksum enables the kernel checksum processing. If on is ture,
// the offset should be an offset in bytes into the data of where the
// checksum field is located.
func (c *dgramOpt) SetChecksum(on bool, offset int) error {
if !c.ok() {
return syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return err
}
return setIPv6Checksum(fd, on, offset)
}
// ICMPFilter returns an ICMP filter.
func (c *dgramOpt) ICMPFilter() (*ICMPFilter, error) {
if !c.ok() {
return nil, syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return nil, err
}
return ipv6ICMPFilter(fd)
}
// SetICMPFilter deploys the ICMP filter.
func (c *dgramOpt) SetICMPFilter(f *ICMPFilter) error {
if !c.ok() {
return syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return err
}
return setIPv6ICMPFilter(fd, f)
}
-95
View File
@@ -1,95 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build dragonfly plan9 solaris
package ipv6
import "net"
// MulticastHopLimit returns the hop limit field value for outgoing
// multicast packets.
func (c *dgramOpt) MulticastHopLimit() (int, error) {
// TODO(mikio): Implement this
return 0, errOpNoSupport
}
// SetMulticastHopLimit sets the hop limit field value for future
// outgoing multicast packets.
func (c *dgramOpt) SetMulticastHopLimit(hoplim int) error {
// TODO(mikio): Implement this
return errOpNoSupport
}
// MulticastInterface returns the default interface for multicast
// packet transmissions.
func (c *dgramOpt) MulticastInterface() (*net.Interface, error) {
// TODO(mikio): Implement this
return nil, errOpNoSupport
}
// SetMulticastInterface sets the default interface for future
// multicast packet transmissions.
func (c *dgramOpt) SetMulticastInterface(ifi *net.Interface) error {
// TODO(mikio): Implement this
return errOpNoSupport
}
// MulticastLoopback reports whether transmitted multicast packets
// should be copied and send back to the originator.
func (c *dgramOpt) MulticastLoopback() (bool, error) {
// TODO(mikio): Implement this
return false, errOpNoSupport
}
// SetMulticastLoopback sets whether transmitted multicast packets
// should be copied and send back to the originator.
func (c *dgramOpt) SetMulticastLoopback(on bool) error {
// TODO(mikio): Implement this
return errOpNoSupport
}
// JoinGroup joins the group address group on the interface ifi.
// It uses the system assigned multicast interface when ifi is nil,
// although this is not recommended because the assignment depends on
// platforms and sometimes it might require routing configuration.
func (c *dgramOpt) JoinGroup(ifi *net.Interface, group net.Addr) error {
// TODO(mikio): Implement this
return errOpNoSupport
}
// LeaveGroup leaves the group address group on the interface ifi.
func (c *dgramOpt) LeaveGroup(ifi *net.Interface, group net.Addr) error {
// TODO(mikio): Implement this
return errOpNoSupport
}
// Checksum reports whether the kernel will compute, store or verify a
// checksum for both incoming and outgoing packets. If on is true, it
// returns an offset in bytes into the data of where the checksum
// field is located.
func (c *dgramOpt) Checksum() (on bool, offset int, err error) {
// TODO(mikio): Implement this
return false, 0, errOpNoSupport
}
// SetChecksum enables the kernel checksum processing. If on is ture,
// the offset should be an offset in bytes into the data of where the
// checksum field is located.
func (c *dgramOpt) SetChecksum(on bool, offset int) error {
// TODO(mikio): Implement this
return errOpNoSupport
}
// ICMPFilter returns an ICMP filter.
func (c *dgramOpt) ICMPFilter() (*ICMPFilter, error) {
// TODO(mikio): Implement this
return nil, errOpNoSupport
}
// SetICMPFilter deploys the ICMP filter.
func (c *dgramOpt) SetICMPFilter(f *ICMPFilter) error {
// TODO(mikio): Implement this
return errOpNoSupport
}
-193
View File
@@ -1,193 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package ipv6 implements IP-level socket options for the Internet
// Protocol version 6.
//
// The package provides IP-level socket options that allow
// manipulation of IPv6 facilities. The IPv6 and socket options for
// IPv6 are defined in RFC 2460, RFC 3493 and RFC 3542.
//
//
// Unicasting
//
// The options for unicasting are available for net.TCPConn,
// net.UDPConn and net.IPConn which are created as network connections
// that use the IPv6 transport. When a single TCP connection carrying
// a data flow of multiple packets needs to indicate the flow is
// important, ipv6.Conn is used to set the traffic class field on the
// IPv6 header for each packet.
//
// ln, err := net.Listen("tcp6", "[::]:1024")
// if err != nil {
// // error handling
// }
// defer ln.Close()
// for {
// c, err := ln.Accept()
// if err != nil {
// // error handling
// }
// go func(c net.Conn) {
// defer c.Close()
//
// The outgoing packets will be labeled DiffServ assured forwarding
// class 1 low drop precedence, as known as AF11 packets.
//
// if err := ipv6.NewConn(c).SetTrafficClass(DiffServAF11); err != nil {
// // error handling
// }
// if _, err := c.Write(data); err != nil {
// // error handling
// }
// }(c)
// }
//
//
// Multicasting
//
// The options for multicasting are available for net.UDPConn and
// net.IPconn which are created as network connections that use the
// IPv6 transport. A few network facilities must be prepared before
// you begin multicasting, at a minimum joining network interfaces and
// multicast groups.
//
// en0, err := net.InterfaceByName("en0")
// if err != nil {
// // error handling
// }
// en1, err := net.InterfaceByIndex(911)
// if err != nil {
// // error handling
// }
// group := net.ParseIP("ff02::114")
//
// First, an application listens to an appropriate address with an
// appropriate service port.
//
// c, err := net.ListenPacket("udp6", "[::]:1024")
// if err != nil {
// // error handling
// }
// defer c.Close()
//
// Second, the application joins multicast groups, starts listening to
// the groups on the specified network interfaces. Note that the
// service port for transport layer protocol does not matter with this
// operation as joining groups affects only network and link layer
// protocols, such as IPv6 and Ethernet.
//
// p := ipv6.NewPacketConn(c)
// if err := p.JoinGroup(en0, &net.UDPAddr{IP: group}); err != nil {
// // error handling
// }
// if err := p.JoinGroup(en1, &net.UDPAddr{IP: group}); err != nil {
// // error handling
// }
//
// The application might set per packet control message transmissions
// between the protocol stack within the kernel. When the application
// needs a destination address on an incoming packet,
// SetControlMessage of ipv6.PacketConn is used to enable control
// message transmissons.
//
// if err := p.SetControlMessage(ipv6.FlagDst, true); err != nil {
// // error handling
// }
//
// The application could identify whether the received packets are
// of interest by using the control message that contains the
// destination address of the received packet.
//
// b := make([]byte, 1500)
// for {
// n, rcm, src, err := p.ReadFrom(b)
// if err != nil {
// // error handling
// }
// if rcm.Dst.IsMulticast() {
// if rcm.Dst.Equal(group)
// // joined group, do something
// } else {
// // unknown group, discard
// continue
// }
// }
//
// The application can also send both unicast and multicast packets.
//
// p.SetTrafficClass(DiffServCS0)
// p.SetHopLimit(16)
// if _, err := p.WriteTo(data[:n], nil, src); err != nil {
// // error handling
// }
// dst := &net.UDPAddr{IP: group, Port: 1024}
// wcm := ipv6.ControlMessage{TrafficClass: DiffServCS7, HopLimit: 1}
// for _, ifi := range []*net.Interface{en0, en1} {
// wcm.IfIndex = ifi.Index
// if _, err := p.WriteTo(data[:n], &wcm, dst); err != nil {
// // error handling
// }
// }
// }
//
//
// More multicasting
//
// An application that uses PacketConn may join multiple multicast
// groups. For example, a UDP listener with port 1024 might join two
// different groups across over two different network interfaces by
// using:
//
// c, err := net.ListenPacket("udp6", "[::]:1024")
// if err != nil {
// // error handling
// }
// defer c.Close()
// p := ipv6.NewPacketConn(c)
// if err := p.JoinGroup(en0, &net.UDPAddr{IP: net.ParseIP("ff02::1:114")}); err != nil {
// // error handling
// }
// if err := p.JoinGroup(en0, &net.UDPAddr{IP: net.ParseIP("ff02::2:114")}); err != nil {
// // error handling
// }
// if err := p.JoinGroup(en1, &net.UDPAddr{IP: net.ParseIP("ff02::2:114")}); err != nil {
// // error handling
// }
//
// It is possible for multiple UDP listeners that listen on the same
// UDP port to join the same multicast group. The net package will
// provide a socket that listens to a wildcard address with reusable
// UDP port when an appropriate multicast address prefix is passed to
// the net.ListenPacket or net.ListenUDP.
//
// c1, err := net.ListenPacket("udp6", "[ff02::]:1024")
// if err != nil {
// // error handling
// }
// defer c1.Close()
// c2, err := net.ListenPacket("udp6", "[ff02::]:1024")
// if err != nil {
// // error handling
// }
// defer c2.Close()
// p1 := ipv6.NewPacketConn(c1)
// if err := p1.JoinGroup(en0, &net.UDPAddr{IP: net.ParseIP("ff02::114")}); err != nil {
// // error handling
// }
// p2 := ipv6.NewPacketConn(c2)
// if err := p2.JoinGroup(en0, &net.UDPAddr{IP: net.ParseIP("ff02::114")}); err != nil {
// // error handling
// }
//
// Also it is possible for the application to leave or rejoin a
// multicast group on the network interface.
//
// if err := p.LeaveGroup(en0, &net.UDPAddr{IP: net.ParseIP("ff02::114")}); err != nil {
// // error handling
// }
// if err := p.JoinGroup(en0, &net.UDPAddr{IP: net.ParseIP("ff01::114")}); err != nil {
// // error handling
// }
package ipv6
-119
View File
@@ -1,119 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"net"
"syscall"
"time"
)
// A Conn represents a network endpoint that uses IPv6 transport.
// It allows to set basic IP-level socket options such as traffic
// class and hop limit.
type Conn struct {
genericOpt
}
type genericOpt struct {
net.Conn
}
func (c *genericOpt) ok() bool { return c != nil && c.Conn != nil }
// PathMTU returns a path MTU value for the destination associated
// with the endpoint.
func (c *Conn) PathMTU() (int, error) {
if !c.genericOpt.ok() {
return 0, syscall.EINVAL
}
fd, err := c.genericOpt.sysfd()
if err != nil {
return 0, err
}
return ipv6PathMTU(fd)
}
// NewConn returns a new Conn.
func NewConn(c net.Conn) *Conn {
return &Conn{
genericOpt: genericOpt{Conn: c},
}
}
// A PacketConn represents a packet network endpoint that uses IPv6
// transport. It is used to control several IP-level socket options
// including IPv6 header manipulation. It also provides datagram
// based network I/O methods specific to the IPv6 and higher layer
// protocols such as OSPF, GRE, and UDP.
type PacketConn struct {
genericOpt
dgramOpt
payloadHandler
}
type dgramOpt struct {
net.PacketConn
}
func (c *dgramOpt) ok() bool { return c != nil && c.PacketConn != nil }
// SetControlMessage allows to receive the per packet basis IP-level
// socket options.
func (c *PacketConn) SetControlMessage(cf ControlFlags, on bool) error {
if !c.payloadHandler.ok() {
return syscall.EINVAL
}
fd, err := c.payloadHandler.sysfd()
if err != nil {
return err
}
return setControlMessage(fd, &c.payloadHandler.rawOpt, cf, on)
}
// SetDeadline sets the read and write deadlines associated with the
// endpoint.
func (c *PacketConn) SetDeadline(t time.Time) error {
if !c.payloadHandler.ok() {
return syscall.EINVAL
}
return c.payloadHandler.SetDeadline(t)
}
// SetReadDeadline sets the read deadline associated with the
// endpoint.
func (c *PacketConn) SetReadDeadline(t time.Time) error {
if !c.payloadHandler.ok() {
return syscall.EINVAL
}
return c.payloadHandler.SetReadDeadline(t)
}
// SetWriteDeadline sets the write deadline associated with the
// endpoint.
func (c *PacketConn) SetWriteDeadline(t time.Time) error {
if !c.payloadHandler.ok() {
return syscall.EINVAL
}
return c.payloadHandler.SetWriteDeadline(t)
}
// Close closes the endpoint.
func (c *PacketConn) Close() error {
if !c.payloadHandler.ok() {
return syscall.EINVAL
}
return c.payloadHandler.Close()
}
// NewPacketConn returns a new PacketConn using c as its underlying
// transport.
func NewPacketConn(c net.PacketConn) *PacketConn {
return &PacketConn{
genericOpt: genericOpt{Conn: c.(net.Conn)},
dgramOpt: dgramOpt{PacketConn: c},
payloadHandler: payloadHandler{PacketConn: c},
}
}
-241
View File
@@ -1,241 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ignore
// This program generates internet protocol constatns and tables by
// reading IANA protocol registries.
//
// Usage:
// go run gen.go > iana.go
package main
import (
"bytes"
"encoding/xml"
"fmt"
"go/format"
"io"
"net/http"
"os"
"strconv"
"strings"
)
var registries = []struct {
url string
parse func(io.Writer, io.Reader) error
}{
{
"http://www.iana.org/assignments/icmpv6-parameters/icmpv6-parameters.xml",
parseICMPv6Parameters,
},
{
"http://www.iana.org/assignments/protocol-numbers/protocol-numbers.xml",
parseProtocolNumbers,
},
}
func main() {
var bb bytes.Buffer
fmt.Fprintf(&bb, "// go run gen.go\n")
fmt.Fprintf(&bb, "// GENERATED BY THE COMMAND ABOVE; DO NOT EDIT\n\n")
fmt.Fprintf(&bb, "package ipv6\n\n")
for _, r := range registries {
resp, err := http.Get(r.url)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
fmt.Fprintf(os.Stderr, "got HTTP status code %v for %v\n", resp.StatusCode, r.url)
os.Exit(1)
}
if err := r.parse(&bb, resp.Body); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Fprintf(&bb, "\n")
}
b, err := format.Source(bb.Bytes())
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
os.Stdout.Write(b)
}
func parseICMPv6Parameters(w io.Writer, r io.Reader) error {
dec := xml.NewDecoder(r)
var icp icmpv6Parameters
if err := dec.Decode(&icp); err != nil {
return err
}
prs := icp.escape()
fmt.Fprintf(w, "// %s, Updated: %s\n", icp.Title, icp.Updated)
fmt.Fprintf(w, "const (\n")
for _, pr := range prs {
if pr.Name == "" {
continue
}
fmt.Fprintf(w, "ICMPType%s ICMPType = %d", pr.Name, pr.Value)
fmt.Fprintf(w, "// %s\n", pr.OrigName)
}
fmt.Fprintf(w, ")\n\n")
fmt.Fprintf(w, "// %s, Updated: %s\n", icp.Title, icp.Updated)
fmt.Fprintf(w, "var icmpTypes = map[ICMPType]string{\n")
for _, pr := range prs {
if pr.Name == "" {
continue
}
fmt.Fprintf(w, "%d: %q,\n", pr.Value, strings.ToLower(pr.OrigName))
}
fmt.Fprintf(w, "}\n")
return nil
}
type icmpv6Parameters struct {
XMLName xml.Name `xml:"registry"`
Title string `xml:"title"`
Updated string `xml:"updated"`
Registries []struct {
Title string `xml:"title"`
Records []struct {
Value string `xml:"value"`
Name string `xml:"name"`
} `xml:"record"`
} `xml:"registry"`
}
type canonICMPv6ParamRecord struct {
OrigName string
Name string
Value int
}
func (icp *icmpv6Parameters) escape() []canonICMPv6ParamRecord {
id := -1
for i, r := range icp.Registries {
if strings.Contains(r.Title, "Type") || strings.Contains(r.Title, "type") {
id = i
break
}
}
if id < 0 {
return nil
}
prs := make([]canonICMPv6ParamRecord, len(icp.Registries[id].Records))
sr := strings.NewReplacer(
"Messages", "",
"Message", "",
"ICMP", "",
"+", "P",
"-", "",
"/", "",
".", "",
" ", "",
)
for i, pr := range icp.Registries[id].Records {
if strings.Contains(pr.Name, "Reserved") ||
strings.Contains(pr.Name, "Unassigned") ||
strings.Contains(pr.Name, "Deprecated") ||
strings.Contains(pr.Name, "Experiment") ||
strings.Contains(pr.Name, "experiment") {
continue
}
ss := strings.Split(pr.Name, "\n")
if len(ss) > 1 {
prs[i].Name = strings.Join(ss, " ")
} else {
prs[i].Name = ss[0]
}
s := strings.TrimSpace(prs[i].Name)
prs[i].OrigName = s
prs[i].Name = sr.Replace(s)
prs[i].Value, _ = strconv.Atoi(pr.Value)
}
return prs
}
func parseProtocolNumbers(w io.Writer, r io.Reader) error {
dec := xml.NewDecoder(r)
var pn protocolNumbers
if err := dec.Decode(&pn); err != nil {
return err
}
prs := pn.escape()
fmt.Fprintf(w, "// %s, Updated: %s\n", pn.Title, pn.Updated)
fmt.Fprintf(w, "const (\n")
for _, pr := range prs {
if pr.Name == "" {
continue
}
fmt.Fprintf(w, "ianaProtocol%s = %d", pr.Name, pr.Value)
s := pr.Descr
if s == "" {
s = pr.OrigName
}
fmt.Fprintf(w, "// %s\n", s)
}
fmt.Fprintf(w, ")\n")
return nil
}
type protocolNumbers struct {
XMLName xml.Name `xml:"registry"`
Title string `xml:"title"`
Updated string `xml:"updated"`
RegTitle string `xml:"registry>title"`
Note string `xml:"registry>note"`
Records []struct {
Value string `xml:"value"`
Name string `xml:"name"`
Descr string `xml:"description"`
} `xml:"registry>record"`
}
type canonProtocolRecord struct {
OrigName string
Name string
Descr string
Value int
}
func (pn *protocolNumbers) escape() []canonProtocolRecord {
prs := make([]canonProtocolRecord, len(pn.Records))
sr := strings.NewReplacer(
"-in-", "in",
"-within-", "within",
"-over-", "over",
"+", "P",
"-", "",
"/", "",
".", "",
" ", "",
)
for i, pr := range pn.Records {
prs[i].OrigName = pr.Name
s := strings.TrimSpace(pr.Name)
switch pr.Name {
case "ISIS over IPv4":
prs[i].Name = "ISIS"
case "manet":
prs[i].Name = "MANET"
default:
prs[i].Name = sr.Replace(s)
}
ss := strings.Split(pr.Descr, "\n")
for i := range ss {
ss[i] = strings.TrimSpace(ss[i])
}
if len(ss) > 1 {
prs[i].Descr = strings.Join(ss, " ")
} else {
prs[i].Descr = ss[0]
}
prs[i].Value, _ = strconv.Atoi(pr.Value)
}
return prs
}
@@ -1,60 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin freebsd linux netbsd openbsd windows
package ipv6
import "syscall"
// TrafficClass returns the traffic class field value for outgoing
// packets.
func (c *genericOpt) TrafficClass() (int, error) {
if !c.ok() {
return 0, syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return 0, err
}
return ipv6TrafficClass(fd)
}
// SetTrafficClass sets the traffic class field value for future
// outgoing packets.
func (c *genericOpt) SetTrafficClass(tclass int) error {
if !c.ok() {
return syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return err
}
return setIPv6TrafficClass(fd, tclass)
}
// HopLimit returns the hop limit field value for outgoing packets.
func (c *genericOpt) HopLimit() (int, error) {
if !c.ok() {
return 0, syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return 0, err
}
return ipv6HopLimit(fd)
}
// SetHopLimit sets the hop limit field value for future outgoing
// packets.
func (c *genericOpt) SetHopLimit(hoplim int) error {
if !c.ok() {
return syscall.EINVAL
}
fd, err := c.sysfd()
if err != nil {
return err
}
return setIPv6HopLimit(fd, hoplim)
}
-34
View File
@@ -1,34 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build dragonfly plan9 solaris
package ipv6
// TrafficClass returns the traffic class field value for outgoing
// packets.
func (c *genericOpt) TrafficClass() (int, error) {
// TODO(mikio): Implement this
return 0, errOpNoSupport
}
// SetTrafficClass sets the traffic class field value for future
// outgoing packets.
func (c *genericOpt) SetTrafficClass(tclass int) error {
// TODO(mikio): Implement this
return errOpNoSupport
}
// HopLimit returns the hop limit field value for outgoing packets.
func (c *genericOpt) HopLimit() (int, error) {
// TODO(mikio): Implement this
return 0, errOpNoSupport
}
// SetHopLimit sets the hop limit field value for future outgoing
// packets.
func (c *genericOpt) SetHopLimit(hoplim int) error {
// TODO(mikio): Implement this
return errOpNoSupport
}
-195
View File
@@ -1,195 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ignore
// This program generates internet protocol constants by reading IANA
// protocol registries.
//
// Usage:
// go run gentest.go > iana_test.go
package main
import (
"bytes"
"encoding/xml"
"fmt"
"go/format"
"io"
"net/http"
"os"
"strconv"
"strings"
)
var registries = []struct {
url string
parse func(io.Writer, io.Reader) error
}{
{
"http://www.iana.org/assignments/dscp-registry/dscp-registry.xml",
parseDSCPRegistry,
},
{
"http://www.iana.org/assignments/ipv4-tos-byte/ipv4-tos-byte.xml",
parseTOSTCByte,
},
}
func main() {
var bb bytes.Buffer
fmt.Fprintf(&bb, "// go run gentest.go\n")
fmt.Fprintf(&bb, "// GENERATED BY THE COMMAND ABOVE; DO NOT EDIT\n\n")
fmt.Fprintf(&bb, "package ipv6_test\n\n")
for _, r := range registries {
resp, err := http.Get(r.url)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
fmt.Fprintf(os.Stderr, "got HTTP status code %v for %v\n", resp.StatusCode, r.url)
os.Exit(1)
}
if err := r.parse(&bb, resp.Body); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Fprintf(&bb, "\n")
}
b, err := format.Source(bb.Bytes())
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
os.Stdout.Write(b)
}
func parseDSCPRegistry(w io.Writer, r io.Reader) error {
dec := xml.NewDecoder(r)
var dr dscpRegistry
if err := dec.Decode(&dr); err != nil {
return err
}
drs := dr.escape()
fmt.Fprintf(w, "// %s, Updated: %s\n", dr.Title, dr.Updated)
fmt.Fprintf(w, "const (\n")
for _, dr := range drs {
fmt.Fprintf(w, "DiffServ%s = %#x", dr.Name, dr.Value)
fmt.Fprintf(w, "// %s\n", dr.OrigName)
}
fmt.Fprintf(w, ")\n")
return nil
}
type dscpRegistry struct {
XMLName xml.Name `xml:"registry"`
Title string `xml:"title"`
Updated string `xml:"updated"`
Note string `xml:"note"`
RegTitle string `xml:"registry>title"`
PoolRecords []struct {
Name string `xml:"name"`
Space string `xml:"space"`
} `xml:"registry>record"`
Records []struct {
Name string `xml:"name"`
Space string `xml:"space"`
} `xml:"registry>registry>record"`
}
type canonDSCPRecord struct {
OrigName string
Name string
Value int
}
func (drr *dscpRegistry) escape() []canonDSCPRecord {
drs := make([]canonDSCPRecord, len(drr.Records))
sr := strings.NewReplacer(
"+", "",
"-", "",
"/", "",
".", "",
" ", "",
)
for i, dr := range drr.Records {
s := strings.TrimSpace(dr.Name)
drs[i].OrigName = s
drs[i].Name = sr.Replace(s)
n, err := strconv.ParseUint(dr.Space, 2, 8)
if err != nil {
continue
}
drs[i].Value = int(n) << 2
}
return drs
}
func parseTOSTCByte(w io.Writer, r io.Reader) error {
dec := xml.NewDecoder(r)
var ttb tosTCByte
if err := dec.Decode(&ttb); err != nil {
return err
}
trs := ttb.escape()
fmt.Fprintf(w, "// %s, Updated: %s\n", ttb.Title, ttb.Updated)
fmt.Fprintf(w, "const (\n")
for _, tr := range trs {
fmt.Fprintf(w, "%s = %#x", tr.Keyword, tr.Value)
fmt.Fprintf(w, "// %s\n", tr.OrigKeyword)
}
fmt.Fprintf(w, ")\n")
return nil
}
type tosTCByte struct {
XMLName xml.Name `xml:"registry"`
Title string `xml:"title"`
Updated string `xml:"updated"`
Note string `xml:"note"`
RegTitle string `xml:"registry>title"`
Records []struct {
Binary string `xml:"binary"`
Keyword string `xml:"keyword"`
} `xml:"registry>record"`
}
type canonTOSTCByteRecord struct {
OrigKeyword string
Keyword string
Value int
}
func (ttb *tosTCByte) escape() []canonTOSTCByteRecord {
trs := make([]canonTOSTCByteRecord, len(ttb.Records))
sr := strings.NewReplacer(
"Capable", "",
"(", "",
")", "",
"+", "",
"-", "",
"/", "",
".", "",
" ", "",
)
for i, tr := range ttb.Records {
s := strings.TrimSpace(tr.Keyword)
trs[i].OrigKeyword = s
ss := strings.Split(s, " ")
if len(ss) > 1 {
trs[i].Keyword = strings.Join(ss[1:], " ")
} else {
trs[i].Keyword = ss[0]
}
trs[i].Keyword = sr.Replace(trs[i].Keyword)
n, err := strconv.ParseUint(tr.Binary, 2, 8)
if err != nil {
continue
}
trs[i].Value = int(n)
}
return trs
}
-33
View File
@@ -1,33 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"errors"
"net"
)
var errOpNoSupport = errors.New("operation not supported")
func boolint(b bool) int {
if b {
return 1
}
return 0
}
func netAddrToIP16(a net.Addr) net.IP {
switch v := a.(type) {
case *net.UDPAddr:
if ip := v.IP.To16(); ip != nil && ip.To4() == nil {
return ip
}
case *net.IPAddr:
if ip := v.IP.To16(); ip != nil && ip.To4() == nil {
return ip
}
}
return nil
}
-22
View File
@@ -1,22 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build dragonfly plan9 solaris
package ipv6
func (c *genericOpt) sysfd() (int, error) {
// TODO(mikio): Implement this
return 0, errOpNoSupport
}
func (c *dgramOpt) sysfd() (int, error) {
// TODO(mikio): Implement this
return 0, errOpNoSupport
}
func (c *payloadHandler) sysfd() (int, error) {
// TODO(mikio): Implement this
return 0, errOpNoSupport
}
-46
View File
@@ -1,46 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin freebsd linux netbsd openbsd
package ipv6
import (
"net"
"reflect"
)
func (c *genericOpt) sysfd() (int, error) {
switch p := c.Conn.(type) {
case *net.TCPConn, *net.UDPConn, *net.IPConn:
return sysfd(p)
}
return 0, errInvalidConnType
}
func (c *dgramOpt) sysfd() (int, error) {
switch p := c.PacketConn.(type) {
case *net.UDPConn, *net.IPConn:
return sysfd(p.(net.Conn))
}
return 0, errInvalidConnType
}
func (c *payloadHandler) sysfd() (int, error) {
return sysfd(c.PacketConn.(net.Conn))
}
func sysfd(c net.Conn) (int, error) {
cv := reflect.ValueOf(c)
switch ce := cv.Elem(); ce.Kind() {
case reflect.Struct:
nfd := ce.FieldByName("conn").FieldByName("fd")
switch fe := nfd.Elem(); fe.Kind() {
case reflect.Struct:
fd := fe.FieldByName("sysfd")
return int(fd.Int()), nil
}
}
return 0, errInvalidConnType
}
-45
View File
@@ -1,45 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"net"
"reflect"
"syscall"
)
func (c *genericOpt) sysfd() (syscall.Handle, error) {
switch p := c.Conn.(type) {
case *net.TCPConn, *net.UDPConn, *net.IPConn:
return sysfd(p)
}
return syscall.InvalidHandle, errInvalidConnType
}
func (c *dgramOpt) sysfd() (syscall.Handle, error) {
switch p := c.PacketConn.(type) {
case *net.UDPConn, *net.IPConn:
return sysfd(p.(net.Conn))
}
return syscall.InvalidHandle, errInvalidConnType
}
func (c *payloadHandler) sysfd() (syscall.Handle, error) {
return sysfd(c.PacketConn.(net.Conn))
}
func sysfd(c net.Conn) (syscall.Handle, error) {
cv := reflect.ValueOf(c)
switch ce := cv.Elem(); ce.Kind() {
case reflect.Struct:
netfd := ce.FieldByName("conn").FieldByName("fd")
switch fe := netfd.Elem(); fe.Kind() {
case reflect.Struct:
fd := fe.FieldByName("sysfd")
return syscall.Handle(fd.Uint()), nil
}
}
return syscall.InvalidHandle, errInvalidConnType
}
-224
View File
@@ -1,224 +0,0 @@
// go run gen.go
// GENERATED BY THE COMMAND ABOVE; DO NOT EDIT
package ipv6
// Internet Control Message Protocol version 6 (ICMPv6) Parameters, Updated: 2013-07-03
const (
ICMPTypeDestinationUnreachable ICMPType = 1 // Destination Unreachable
ICMPTypePacketTooBig ICMPType = 2 // Packet Too Big
ICMPTypeTimeExceeded ICMPType = 3 // Time Exceeded
ICMPTypeParameterProblem ICMPType = 4 // Parameter Problem
ICMPTypeEchoRequest ICMPType = 128 // Echo Request
ICMPTypeEchoReply ICMPType = 129 // Echo Reply
ICMPTypeMulticastListenerQuery ICMPType = 130 // Multicast Listener Query
ICMPTypeMulticastListenerReport ICMPType = 131 // Multicast Listener Report
ICMPTypeMulticastListenerDone ICMPType = 132 // Multicast Listener Done
ICMPTypeRouterSolicitation ICMPType = 133 // Router Solicitation
ICMPTypeRouterAdvertisement ICMPType = 134 // Router Advertisement
ICMPTypeNeighborSolicitation ICMPType = 135 // Neighbor Solicitation
ICMPTypeNeighborAdvertisement ICMPType = 136 // Neighbor Advertisement
ICMPTypeRedirect ICMPType = 137 // Redirect Message
ICMPTypeRouterRenumbering ICMPType = 138 // Router Renumbering
ICMPTypeNodeInformationQuery ICMPType = 139 // ICMP Node Information Query
ICMPTypeNodeInformationResponse ICMPType = 140 // ICMP Node Information Response
ICMPTypeInverseNeighborDiscoverySolicitation ICMPType = 141 // Inverse Neighbor Discovery Solicitation Message
ICMPTypeInverseNeighborDiscoveryAdvertisement ICMPType = 142 // Inverse Neighbor Discovery Advertisement Message
ICMPTypeVersion2MulticastListenerReport ICMPType = 143 // Version 2 Multicast Listener Report
ICMPTypeHomeAgentAddressDiscoveryRequest ICMPType = 144 // Home Agent Address Discovery Request Message
ICMPTypeHomeAgentAddressDiscoveryReply ICMPType = 145 // Home Agent Address Discovery Reply Message
ICMPTypeMobilePrefixSolicitation ICMPType = 146 // Mobile Prefix Solicitation
ICMPTypeMobilePrefixAdvertisement ICMPType = 147 // Mobile Prefix Advertisement
ICMPTypeCertificationPathSolicitation ICMPType = 148 // Certification Path Solicitation Message
ICMPTypeCertificationPathAdvertisement ICMPType = 149 // Certification Path Advertisement Message
ICMPTypeMulticastRouterAdvertisement ICMPType = 151 // Multicast Router Advertisement
ICMPTypeMulticastRouterSolicitation ICMPType = 152 // Multicast Router Solicitation
ICMPTypeMulticastRouterTermination ICMPType = 153 // Multicast Router Termination
ICMPTypeFMIPv6 ICMPType = 154 // FMIPv6 Messages
ICMPTypeRPLControl ICMPType = 155 // RPL Control Message
ICMPTypeILNPv6LocatorUpdate ICMPType = 156 // ILNPv6 Locator Update Message
ICMPTypeDuplicateAddressRequest ICMPType = 157 // Duplicate Address Request
ICMPTypeDuplicateAddressConfirmation ICMPType = 158 // Duplicate Address Confirmation
)
// Internet Control Message Protocol version 6 (ICMPv6) Parameters, Updated: 2013-07-03
var icmpTypes = map[ICMPType]string{
1: "destination unreachable",
2: "packet too big",
3: "time exceeded",
4: "parameter problem",
128: "echo request",
129: "echo reply",
130: "multicast listener query",
131: "multicast listener report",
132: "multicast listener done",
133: "router solicitation",
134: "router advertisement",
135: "neighbor solicitation",
136: "neighbor advertisement",
137: "redirect message",
138: "router renumbering",
139: "icmp node information query",
140: "icmp node information response",
141: "inverse neighbor discovery solicitation message",
142: "inverse neighbor discovery advertisement message",
143: "version 2 multicast listener report",
144: "home agent address discovery request message",
145: "home agent address discovery reply message",
146: "mobile prefix solicitation",
147: "mobile prefix advertisement",
148: "certification path solicitation message",
149: "certification path advertisement message",
151: "multicast router advertisement",
152: "multicast router solicitation",
153: "multicast router termination",
154: "fmipv6 messages",
155: "rpl control message",
156: "ilnpv6 locator update message",
157: "duplicate address request",
158: "duplicate address confirmation",
}
// Protocol Numbers, Updated: 2013-02-17
const (
ianaProtocolHOPOPT = 0 // IPv6 Hop-by-Hop Option
ianaProtocolICMP = 1 // Internet Control Message
ianaProtocolIGMP = 2 // Internet Group Management
ianaProtocolGGP = 3 // Gateway-to-Gateway
ianaProtocolIPv4 = 4 // IPv4 encapsulation
ianaProtocolST = 5 // Stream
ianaProtocolTCP = 6 // Transmission Control
ianaProtocolCBT = 7 // CBT
ianaProtocolEGP = 8 // Exterior Gateway Protocol
ianaProtocolIGP = 9 // any private interior gateway (used by Cisco for their IGRP)
ianaProtocolBBNRCCMON = 10 // BBN RCC Monitoring
ianaProtocolNVPII = 11 // Network Voice Protocol
ianaProtocolPUP = 12 // PUP
ianaProtocolARGUS = 13 // ARGUS
ianaProtocolEMCON = 14 // EMCON
ianaProtocolXNET = 15 // Cross Net Debugger
ianaProtocolCHAOS = 16 // Chaos
ianaProtocolUDP = 17 // User Datagram
ianaProtocolMUX = 18 // Multiplexing
ianaProtocolDCNMEAS = 19 // DCN Measurement Subsystems
ianaProtocolHMP = 20 // Host Monitoring
ianaProtocolPRM = 21 // Packet Radio Measurement
ianaProtocolXNSIDP = 22 // XEROX NS IDP
ianaProtocolTRUNK1 = 23 // Trunk-1
ianaProtocolTRUNK2 = 24 // Trunk-2
ianaProtocolLEAF1 = 25 // Leaf-1
ianaProtocolLEAF2 = 26 // Leaf-2
ianaProtocolRDP = 27 // Reliable Data Protocol
ianaProtocolIRTP = 28 // Internet Reliable Transaction
ianaProtocolISOTP4 = 29 // ISO Transport Protocol Class 4
ianaProtocolNETBLT = 30 // Bulk Data Transfer Protocol
ianaProtocolMFENSP = 31 // MFE Network Services Protocol
ianaProtocolMERITINP = 32 // MERIT Internodal Protocol
ianaProtocolDCCP = 33 // Datagram Congestion Control Protocol
ianaProtocol3PC = 34 // Third Party Connect Protocol
ianaProtocolIDPR = 35 // Inter-Domain Policy Routing Protocol
ianaProtocolXTP = 36 // XTP
ianaProtocolDDP = 37 // Datagram Delivery Protocol
ianaProtocolIDPRCMTP = 38 // IDPR Control Message Transport Proto
ianaProtocolTPPP = 39 // TP++ Transport Protocol
ianaProtocolIL = 40 // IL Transport Protocol
ianaProtocolIPv6 = 41 // IPv6 encapsulation
ianaProtocolSDRP = 42 // Source Demand Routing Protocol
ianaProtocolIPv6Route = 43 // Routing Header for IPv6
ianaProtocolIPv6Frag = 44 // Fragment Header for IPv6
ianaProtocolIDRP = 45 // Inter-Domain Routing Protocol
ianaProtocolRSVP = 46 // Reservation Protocol
ianaProtocolGRE = 47 // Generic Routing Encapsulation
ianaProtocolDSR = 48 // Dynamic Source Routing Protocol
ianaProtocolBNA = 49 // BNA
ianaProtocolESP = 50 // Encap Security Payload
ianaProtocolAH = 51 // Authentication Header
ianaProtocolINLSP = 52 // Integrated Net Layer Security TUBA
ianaProtocolSWIPE = 53 // IP with Encryption
ianaProtocolNARP = 54 // NBMA Address Resolution Protocol
ianaProtocolMOBILE = 55 // IP Mobility
ianaProtocolTLSP = 56 // Transport Layer Security Protocol using Kryptonet key management
ianaProtocolSKIP = 57 // SKIP
ianaProtocolIPv6ICMP = 58 // ICMP for IPv6
ianaProtocolIPv6NoNxt = 59 // No Next Header for IPv6
ianaProtocolIPv6Opts = 60 // Destination Options for IPv6
ianaProtocolCFTP = 62 // CFTP
ianaProtocolSATEXPAK = 64 // SATNET and Backroom EXPAK
ianaProtocolKRYPTOLAN = 65 // Kryptolan
ianaProtocolRVD = 66 // MIT Remote Virtual Disk Protocol
ianaProtocolIPPC = 67 // Internet Pluribus Packet Core
ianaProtocolSATMON = 69 // SATNET Monitoring
ianaProtocolVISA = 70 // VISA Protocol
ianaProtocolIPCV = 71 // Internet Packet Core Utility
ianaProtocolCPNX = 72 // Computer Protocol Network Executive
ianaProtocolCPHB = 73 // Computer Protocol Heart Beat
ianaProtocolWSN = 74 // Wang Span Network
ianaProtocolPVP = 75 // Packet Video Protocol
ianaProtocolBRSATMON = 76 // Backroom SATNET Monitoring
ianaProtocolSUNND = 77 // SUN ND PROTOCOL-Temporary
ianaProtocolWBMON = 78 // WIDEBAND Monitoring
ianaProtocolWBEXPAK = 79 // WIDEBAND EXPAK
ianaProtocolISOIP = 80 // ISO Internet Protocol
ianaProtocolVMTP = 81 // VMTP
ianaProtocolSECUREVMTP = 82 // SECURE-VMTP
ianaProtocolVINES = 83 // VINES
ianaProtocolTTP = 84 // TTP
ianaProtocolIPTM = 84 // Protocol Internet Protocol Traffic Manager
ianaProtocolNSFNETIGP = 85 // NSFNET-IGP
ianaProtocolDGP = 86 // Dissimilar Gateway Protocol
ianaProtocolTCF = 87 // TCF
ianaProtocolEIGRP = 88 // EIGRP
ianaProtocolOSPFIGP = 89 // OSPFIGP
ianaProtocolSpriteRPC = 90 // Sprite RPC Protocol
ianaProtocolLARP = 91 // Locus Address Resolution Protocol
ianaProtocolMTP = 92 // Multicast Transport Protocol
ianaProtocolAX25 = 93 // AX.25 Frames
ianaProtocolIPIP = 94 // IP-within-IP Encapsulation Protocol
ianaProtocolMICP = 95 // Mobile Internetworking Control Pro.
ianaProtocolSCCSP = 96 // Semaphore Communications Sec. Pro.
ianaProtocolETHERIP = 97 // Ethernet-within-IP Encapsulation
ianaProtocolENCAP = 98 // Encapsulation Header
ianaProtocolGMTP = 100 // GMTP
ianaProtocolIFMP = 101 // Ipsilon Flow Management Protocol
ianaProtocolPNNI = 102 // PNNI over IP
ianaProtocolPIM = 103 // Protocol Independent Multicast
ianaProtocolARIS = 104 // ARIS
ianaProtocolSCPS = 105 // SCPS
ianaProtocolQNX = 106 // QNX
ianaProtocolAN = 107 // Active Networks
ianaProtocolIPComp = 108 // IP Payload Compression Protocol
ianaProtocolSNP = 109 // Sitara Networks Protocol
ianaProtocolCompaqPeer = 110 // Compaq Peer Protocol
ianaProtocolIPXinIP = 111 // IPX in IP
ianaProtocolVRRP = 112 // Virtual Router Redundancy Protocol
ianaProtocolPGM = 113 // PGM Reliable Transport Protocol
ianaProtocolL2TP = 115 // Layer Two Tunneling Protocol
ianaProtocolDDX = 116 // D-II Data Exchange (DDX)
ianaProtocolIATP = 117 // Interactive Agent Transfer Protocol
ianaProtocolSTP = 118 // Schedule Transfer Protocol
ianaProtocolSRP = 119 // SpectraLink Radio Protocol
ianaProtocolUTI = 120 // UTI
ianaProtocolSMP = 121 // Simple Message Protocol
ianaProtocolSM = 122 // SM
ianaProtocolPTP = 123 // Performance Transparency Protocol
ianaProtocolISIS = 124 // ISIS over IPv4
ianaProtocolFIRE = 125 // FIRE
ianaProtocolCRTP = 126 // Combat Radio Transport Protocol
ianaProtocolCRUDP = 127 // Combat Radio User Datagram
ianaProtocolSSCOPMCE = 128 // SSCOPMCE
ianaProtocolIPLT = 129 // IPLT
ianaProtocolSPS = 130 // Secure Packet Shield
ianaProtocolPIPE = 131 // Private IP Encapsulation within IP
ianaProtocolSCTP = 132 // Stream Control Transmission Protocol
ianaProtocolFC = 133 // Fibre Channel
ianaProtocolRSVPE2EIGNORE = 134 // RSVP-E2E-IGNORE
ianaProtocolMobilityHeader = 135 // Mobility Header
ianaProtocolUDPLite = 136 // UDPLite
ianaProtocolMPLSinIP = 137 // MPLS-in-IP
ianaProtocolMANET = 138 // MANET Protocols
ianaProtocolHIP = 139 // Host Identity Protocol
ianaProtocolShim6 = 140 // Shim6 Protocol
ianaProtocolWESP = 141 // Wrapped Encapsulating Security Payload
ianaProtocolROHC = 142 // Robust Header Compression
ianaProtocolReserved = 255 // Reserved
)
-38
View File
@@ -1,38 +0,0 @@
// go run gentest.go
// GENERATED BY THE COMMAND ABOVE; DO NOT EDIT
package ipv6_test
// Differentiated Services Field Codepoints (DSCP), Updated: 2013-06-25
const (
DiffServCS0 = 0x0 // CS0
DiffServCS1 = 0x20 // CS1
DiffServCS2 = 0x40 // CS2
DiffServCS3 = 0x60 // CS3
DiffServCS4 = 0x80 // CS4
DiffServCS5 = 0xa0 // CS5
DiffServCS6 = 0xc0 // CS6
DiffServCS7 = 0xe0 // CS7
DiffServAF11 = 0x28 // AF11
DiffServAF12 = 0x30 // AF12
DiffServAF13 = 0x38 // AF13
DiffServAF21 = 0x48 // AF21
DiffServAF22 = 0x50 // AF22
DiffServAF23 = 0x58 // AF23
DiffServAF31 = 0x68 // AF31
DiffServAF32 = 0x70 // AF32
DiffServAF33 = 0x78 // AF33
DiffServAF41 = 0x88 // AF41
DiffServAF42 = 0x90 // AF42
DiffServAF43 = 0x98 // AF43
DiffServEFPHB = 0xb8 // EF PHB
DiffServVOICEADMIT = 0xb0 // VOICE-ADMIT
)
// IPv4 TOS Byte and IPv6 Traffic Class Octet, Updated: 2001-09-06
const (
NotECNTransport = 0x0 // Not-ECT (Not ECN-Capable Transport)
ECNTransport1 = 0x1 // ECT(1) (ECN-Capable Transport(1))
ECNTransport0 = 0x2 // ECT(0) (ECN-Capable Transport(0))
CongestionExperienced = 0x3 // CE (Congestion Experienced)
)
-47
View File
@@ -1,47 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import "sync"
// An ICMPType represents a type of ICMP message.
type ICMPType int
func (typ ICMPType) String() string {
s, ok := icmpTypes[typ]
if !ok {
return "<nil>"
}
return s
}
// An ICMPFilter represents an ICMP message filter for incoming
// packets.
type ICMPFilter struct {
mu sync.RWMutex
sysICMPFilter
}
// Set sets the ICMP type and filter action to the filter.
func (f *ICMPFilter) Set(typ ICMPType, block bool) {
f.mu.Lock()
f.set(typ, block)
f.mu.Unlock()
}
// SetAll sets the filter action to the filter.
func (f *ICMPFilter) SetAll(block bool) {
f.mu.Lock()
f.setAll(block)
f.mu.Unlock()
}
// WillBlock reports whether the ICMP type will be blocked.
func (f *ICMPFilter) WillBlock(typ ICMPType) bool {
f.mu.RLock()
ok := f.willBlock(typ)
f.mu.RUnlock()
return ok
}
-33
View File
@@ -1,33 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin freebsd netbsd openbsd
package ipv6
type sysICMPFilter struct {
Filt [8]uint32
}
func (f *sysICMPFilter) set(typ ICMPType, block bool) {
if block {
f.Filt[typ>>5] &^= 1 << (uint32(typ) & 31)
} else {
f.Filt[typ>>5] |= 1 << (uint32(typ) & 31)
}
}
func (f *sysICMPFilter) setAll(block bool) {
for i := range f.Filt {
if block {
f.Filt[i] = 0
} else {
f.Filt[i] = 1<<32 - 1
}
}
}
func (f *sysICMPFilter) willBlock(typ ICMPType) bool {
return f.Filt[typ>>5]&(1<<(uint32(typ)&31)) == 0
}
-31
View File
@@ -1,31 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
type sysICMPFilter struct {
Data [8]uint32
}
func (f *sysICMPFilter) set(typ ICMPType, block bool) {
if block {
f.Data[typ>>5] |= 1 << (uint32(typ) & 31)
} else {
f.Data[typ>>5] &^= 1 << (uint32(typ) & 31)
}
}
func (f *sysICMPFilter) setAll(block bool) {
for i := range f.Data {
if block {
f.Data[i] = 1<<32 - 1
} else {
f.Data[i] = 0
}
}
}
func (f *sysICMPFilter) willBlock(typ ICMPType) bool {
return f.Data[typ>>5]&(1<<(uint32(typ)&31)) != 0
}
-24
View File
@@ -1,24 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build dragonfly plan9 solaris
package ipv6
type sysICMPFilter struct {
// TODO(mikio): Implement this
}
func (f *sysICMPFilter) set(typ ICMPType, block bool) {
// TODO(mikio): Implement this
}
func (f *sysICMPFilter) setAll(block bool) {
// TODO(mikio): Implement this
}
func (f *sysICMPFilter) willBlock(typ ICMPType) bool {
// TODO(mikio): Implement this
return false
}
-102
View File
@@ -1,102 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6_test
import (
"code.google.com/p/go.net/ipv6"
"net"
"os"
"reflect"
"runtime"
"sync"
"testing"
)
var icmpStringTests = []struct {
in ipv6.ICMPType
out string
}{
{ipv6.ICMPTypeDestinationUnreachable, "destination unreachable"},
{256, "<nil>"},
}
func TestICMPString(t *testing.T) {
for _, tt := range icmpStringTests {
s := tt.in.String()
if s != tt.out {
t.Errorf("got %s; expected %s", s, tt.out)
}
}
}
func TestICMPFilter(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
var f ipv6.ICMPFilter
for _, toggle := range []bool{false, true} {
f.SetAll(toggle)
var wg sync.WaitGroup
for _, typ := range []ipv6.ICMPType{
ipv6.ICMPTypeDestinationUnreachable,
ipv6.ICMPTypeEchoReply,
ipv6.ICMPTypeNeighborSolicitation,
ipv6.ICMPTypeDuplicateAddressConfirmation,
} {
wg.Add(1)
go func(typ ipv6.ICMPType) {
defer wg.Done()
f.Set(typ, false)
if f.WillBlock(typ) {
t.Errorf("ipv6.ICMPFilter.Set(%v, false) failed", typ)
}
f.Set(typ, true)
if !f.WillBlock(typ) {
t.Errorf("ipv6.ICMPFilter.Set(%v, true) failed", typ)
}
}(typ)
}
wg.Wait()
}
}
func TestSetICMPFilter(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
if os.Getuid() != 0 {
t.Skip("must be root")
}
c, err := net.ListenPacket("ip6:ipv6-icmp", "::1")
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c.Close()
p := ipv6.NewPacketConn(c)
var f ipv6.ICMPFilter
f.SetAll(true)
f.Set(ipv6.ICMPTypeEchoRequest, false)
f.Set(ipv6.ICMPTypeEchoReply, false)
if err := p.SetICMPFilter(&f); err != nil {
t.Fatalf("ipv6.PacketConn.SetICMPFilter failed: %v", err)
}
kf, err := p.ICMPFilter()
if err != nil {
t.Fatalf("ipv6.PacketConn.ICMPFilter failed: %v", err)
}
if !reflect.DeepEqual(kf, &f) {
t.Fatalf("got unexpected filter %#v; expected %#v", kf, f)
}
}
-22
View File
@@ -1,22 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
type sysICMPFilter struct {
// TODO(mikio): Implement this
}
func (f *sysICMPFilter) set(typ ICMPType, block bool) {
// TODO(mikio): Implement this
}
func (f *sysICMPFilter) setAll(block bool) {
// TODO(mikio): Implement this
}
func (f *sysICMPFilter) willBlock(typ ICMPType) bool {
// TODO(mikio): Implement this
return false
}
-130
View File
@@ -1,130 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6_test
import (
"code.google.com/p/go.net/ipv6"
"errors"
"net"
)
const (
ipv6PseudoHeaderLen = 2*net.IPv6len + 8
ianaProtocolIPv6ICMP = 58
)
func ipv6PseudoHeader(src, dst net.IP, nextHeader int) []byte {
b := make([]byte, ipv6PseudoHeaderLen)
copy(b[:net.IPv6len], src)
copy(b[net.IPv6len:], dst)
b[len(b)-1] = byte(nextHeader)
return b
}
// icmpMessage represents an ICMP message.
type icmpMessage struct {
Type ipv6.ICMPType // type
Code int // code
Checksum int // checksum
Body icmpMessageBody // body
}
// icmpMessageBody represents an ICMP message body.
type icmpMessageBody interface {
Len() int
Marshal() ([]byte, error)
}
// Marshal returns the binary enconding of the ICMP echo request or
// reply message m.
func (m *icmpMessage) Marshal(psh []byte) ([]byte, error) {
b := []byte{byte(m.Type), byte(m.Code), 0, 0}
if psh != nil {
b = append(psh, b...)
}
if m.Body != nil && m.Body.Len() != 0 {
mb, err := m.Body.Marshal()
if err != nil {
return nil, err
}
b = append(b, mb...)
}
if psh == nil {
return b, nil
}
off, l := 2*net.IPv6len, len(b)-len(psh)
b[off], b[off+1], b[off+2], b[off+3] = byte(l>>24), byte(l>>16), byte(l>>8), byte(l)
csumcv := len(b) - 1 // checksum coverage
s := uint32(0)
for i := 0; i < csumcv; i += 2 {
s += uint32(b[i+1])<<8 | uint32(b[i])
}
if csumcv&1 == 0 {
s += uint32(b[csumcv])
}
s = s>>16 + s&0xffff
s = s + s>>16
// Place checksum back in header; using ^= avoids the
// assumption the checksum bytes are zero.
b[len(psh)+2] ^= byte(^s)
b[len(psh)+3] ^= byte(^s >> 8)
return b[len(psh):], nil
}
// parseICMPMessage parses b as an ICMP message.
func parseICMPMessage(b []byte) (*icmpMessage, error) {
msglen := len(b)
if msglen < 4 {
return nil, errors.New("message too short")
}
m := &icmpMessage{Type: ipv6.ICMPType(b[0]), Code: int(b[1]), Checksum: int(b[2])<<8 | int(b[3])}
if msglen > 4 {
var err error
switch m.Type {
case ipv6.ICMPTypeEchoRequest, ipv6.ICMPTypeEchoReply:
m.Body, err = parseICMPEcho(b[4:])
if err != nil {
return nil, err
}
}
}
return m, nil
}
// imcpEcho represenets an ICMP echo request or reply message body.
type icmpEcho struct {
ID int // identifier
Seq int // sequence number
Data []byte // data
}
func (p *icmpEcho) Len() int {
if p == nil {
return 0
}
return 4 + len(p.Data)
}
// Marshal returns the binary enconding of the ICMP echo request or
// reply message body p.
func (p *icmpEcho) Marshal() ([]byte, error) {
b := make([]byte, 4+len(p.Data))
b[0], b[1] = byte(p.ID>>8), byte(p.ID)
b[2], b[3] = byte(p.Seq>>8), byte(p.Seq)
copy(b[4:], p.Data)
return b, nil
}
// parseICMPEcho parses b as an ICMP echo request or reply message
// body.
func parseICMPEcho(b []byte) (*icmpEcho, error) {
bodylen := len(b)
p := &icmpEcho{ID: int(b[0])<<8 | int(b[1]), Seq: int(b[2])<<8 | int(b[3])}
if bodylen > 4 {
p.Data = make([]byte, bodylen-4)
copy(p.Data, b[4:])
}
return p, nil
}
@@ -1,88 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6_test
import (
"net"
"testing"
)
func isLinkLocalUnicast(ip net.IP) bool {
return ip.To4() == nil && ip.To16() != nil && ip.IsLinkLocalUnicast()
}
func loopbackInterface() *net.Interface {
ift, err := net.Interfaces()
if err != nil {
return nil
}
for _, ifi := range ift {
if ifi.Flags&net.FlagLoopback == 0 || ifi.Flags&net.FlagUp == 0 {
continue
}
ifat, err := ifi.Addrs()
if err != nil {
continue
}
for _, ifa := range ifat {
switch ifa := ifa.(type) {
case *net.IPAddr:
if isLinkLocalUnicast(ifa.IP) {
return &ifi
}
case *net.IPNet:
if isLinkLocalUnicast(ifa.IP) {
return &ifi
}
}
}
}
return nil
}
func isMulticastAvailable(ifi *net.Interface) (net.IP, bool) {
if ifi == nil || ifi.Flags&net.FlagUp == 0 || ifi.Flags&net.FlagMulticast == 0 {
return nil, false
}
ifat, err := ifi.Addrs()
if err != nil {
return nil, false
}
for _, ifa := range ifat {
switch ifa := ifa.(type) {
case *net.IPAddr:
if isLinkLocalUnicast(ifa.IP) {
return ifa.IP, true
}
case *net.IPNet:
if isLinkLocalUnicast(ifa.IP) {
return ifa.IP, true
}
}
}
return nil, false
}
func connector(t *testing.T, network, addr string, done chan<- bool) {
defer func() { done <- true }()
c, err := net.Dial(network, addr)
if err != nil {
t.Errorf("net.Dial failed: %v", err)
return
}
c.Close()
}
func acceptor(t *testing.T, ln net.Listener, done chan<- bool) {
defer func() { done <- true }()
c, err := ln.Accept()
if err != nil {
t.Errorf("net.Listener.Accept failed: %v", err)
return
}
c.Close()
}
-205
View File
@@ -1,205 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6_test
import (
"bytes"
"code.google.com/p/go.net/ipv6"
"net"
"os"
"runtime"
"testing"
"time"
)
func TestPacketConnReadWriteMulticastUDP(t *testing.T) {
switch runtime.GOOS {
case "freebsd": // due to a bug on loopback marking
// See http://www.freebsd.org/cgi/query-pr.cgi?pr=180065.
t.Skipf("not supported on %q", runtime.GOOS)
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
ifi := loopbackInterface()
if ifi == nil {
t.Skipf("not available on %q", runtime.GOOS)
}
c, err := net.ListenPacket("udp6", "[ff02::114]:0") // see RFC 4727
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c.Close()
_, port, err := net.SplitHostPort(c.LocalAddr().String())
if err != nil {
t.Fatalf("net.SplitHostPort failed: %v", err)
}
dst, err := net.ResolveUDPAddr("udp6", "[ff02::114]:"+port) // see RFC 4727
if err != nil {
t.Fatalf("net.ResolveUDPAddr failed: %v", err)
}
p := ipv6.NewPacketConn(c)
defer p.Close()
if err := p.JoinGroup(ifi, dst); err != nil {
t.Fatalf("ipv6.PacketConn.JoinGroup on %v failed: %v", ifi, err)
}
if err := p.SetMulticastInterface(ifi); err != nil {
t.Fatalf("ipv6.PacketConn.SetMulticastInterface failed: %v", err)
}
if _, err := p.MulticastInterface(); err != nil {
t.Fatalf("ipv6.PacketConn.MulticastInterface failed: %v", err)
}
if err := p.SetMulticastLoopback(true); err != nil {
t.Fatalf("ipv6.PacketConn.SetMulticastLoopback failed: %v", err)
}
if _, err := p.MulticastLoopback(); err != nil {
t.Fatalf("ipv6.PacketConn.MulticastLoopback failed: %v", err)
}
cm := ipv6.ControlMessage{
TrafficClass: DiffServAF11 | CongestionExperienced,
Src: net.IPv6loopback,
IfIndex: ifi.Index,
}
cf := ipv6.FlagTrafficClass | ipv6.FlagHopLimit | ipv6.FlagSrc | ipv6.FlagDst | ipv6.FlagInterface | ipv6.FlagPathMTU
wb := []byte("HELLO-R-U-THERE")
for i, toggle := range []bool{true, false, true} {
if err := p.SetControlMessage(cf, toggle); err != nil {
t.Fatalf("ipv6.PacketConn.SetControlMessage failed: %v", err)
}
if err := p.SetDeadline(time.Now().Add(200 * time.Millisecond)); err != nil {
t.Fatalf("ipv6.PacketConn.SetDeadline failed: %v", err)
}
cm.HopLimit = i + 1
if n, err := p.WriteTo(wb, &cm, dst); err != nil {
t.Fatalf("ipv6.PacketConn.WriteTo failed: %v", err)
} else if n != len(wb) {
t.Fatalf("ipv6.PacketConn.WriteTo failed: short write: %v", n)
}
rb := make([]byte, 128)
if n, cm, _, err := p.ReadFrom(rb); err != nil {
t.Fatalf("ipv6.PacketConn.ReadFrom failed: %v", err)
} else if !bytes.Equal(rb[:n], wb) {
t.Fatalf("got %v; expected %v", rb[:n], wb)
} else {
t.Logf("rcvd cmsg: %v", cm)
}
}
}
func TestPacketConnReadWriteMulticastICMP(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
if os.Getuid() != 0 {
t.Skip("must be root")
}
ifi := loopbackInterface()
if ifi == nil {
t.Skipf("not available on %q", runtime.GOOS)
}
c, err := net.ListenPacket("ip6:ipv6-icmp", "::")
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c.Close()
dst, err := net.ResolveIPAddr("ip6", "ff02::114") // see RFC 4727
if err != nil {
t.Fatalf("net.ResolveIPAddr failed: %v", err)
}
pshicmp := ipv6PseudoHeader(c.LocalAddr().(*net.IPAddr).IP, dst.IP, ianaProtocolIPv6ICMP)
p := ipv6.NewPacketConn(c)
defer p.Close()
if err := p.JoinGroup(ifi, dst); err != nil {
t.Fatalf("ipv6.PacketConn.JoinGroup on %v failed: %v", ifi, err)
}
if err := p.SetMulticastInterface(ifi); err != nil {
t.Fatalf("ipv6.PacketConn.SetMulticastInterface failed: %v", err)
}
if _, err := p.MulticastInterface(); err != nil {
t.Fatalf("ipv6.PacketConn.MulticastInterface failed: %v", err)
}
if err := p.SetMulticastLoopback(true); err != nil {
t.Fatalf("ipv6.PacketConn.SetMulticastLoopback failed: %v", err)
}
if _, err := p.MulticastLoopback(); err != nil {
t.Fatalf("ipv6.PacketConn.MulticastLoopback failed: %v", err)
}
cm := ipv6.ControlMessage{
TrafficClass: DiffServAF11 | CongestionExperienced,
Src: net.IPv6loopback,
IfIndex: ifi.Index,
}
cf := ipv6.FlagTrafficClass | ipv6.FlagHopLimit | ipv6.FlagSrc | ipv6.FlagDst | ipv6.FlagInterface | ipv6.FlagPathMTU
var f ipv6.ICMPFilter
f.SetAll(true)
f.Set(ipv6.ICMPTypeEchoReply, false)
if err := p.SetICMPFilter(&f); err != nil {
t.Fatalf("ipv6.PacketConn.SetICMPFilter failed: %v", err)
}
var psh []byte
for i, toggle := range []bool{true, false, true} {
if toggle {
psh = nil
if err := p.SetChecksum(true, 2); err != nil {
t.Fatalf("ipv6.PacketConn.SetChecksum failed: %v", err)
}
} else {
psh = pshicmp
// Some platforms never allow to disable the
// kernel checksum processing.
p.SetChecksum(false, -1)
}
wb, err := (&icmpMessage{
Type: ipv6.ICMPTypeEchoRequest, Code: 0,
Body: &icmpEcho{
ID: os.Getpid() & 0xffff, Seq: i + 1,
Data: []byte("HELLO-R-U-THERE"),
},
}).Marshal(psh)
if err != nil {
t.Fatalf("icmpMessage.Marshal failed: %v", err)
}
if err := p.SetControlMessage(cf, toggle); err != nil {
t.Fatalf("ipv6.PacketConn.SetControlMessage failed: %v", err)
}
if err := p.SetDeadline(time.Now().Add(200 * time.Millisecond)); err != nil {
t.Fatalf("ipv6.PacketConn.SetDeadline failed: %v", err)
}
cm.HopLimit = i + 1
if n, err := p.WriteTo(wb, &cm, dst); err != nil {
t.Fatalf("ipv6.PacketConn.WriteTo failed: %v", err)
} else if n != len(wb) {
t.Fatalf("ipv6.PacketConn.WriteTo failed: short write: %v", n)
}
rb := make([]byte, 128)
if n, cm, _, err := p.ReadFrom(rb); err != nil {
t.Fatalf("ipv6.PacketConn.ReadFrom failed: %v", err)
} else {
t.Logf("rcvd cmsg: %v", cm)
if m, err := parseICMPMessage(rb[:n]); err != nil {
t.Fatalf("parseICMPMessage failed: %v", err)
} else if m.Type != ipv6.ICMPTypeEchoReply || m.Code != 0 {
t.Fatalf("got type=%v, code=%v; expected type=%v, code=%v", m.Type, m.Code, ipv6.ICMPTypeEchoReply, 0)
}
}
}
}
@@ -1,243 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6_test
import (
"code.google.com/p/go.net/ipv6"
"fmt"
"net"
"os"
"runtime"
"testing"
)
var udpMultipleGroupListenerTests = []net.Addr{
&net.UDPAddr{IP: net.ParseIP("ff02::114")}, // see RFC 4727
&net.UDPAddr{IP: net.ParseIP("ff02::1:114")},
&net.UDPAddr{IP: net.ParseIP("ff02::2:114")},
}
func TestUDPSinglePacketConnWithMultipleGroupListeners(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
for _, gaddr := range udpMultipleGroupListenerTests {
c, err := net.ListenPacket("udp6", "[::]:0") // wildcard address with non-reusable port
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c.Close()
p := ipv6.NewPacketConn(c)
var mift []*net.Interface
ift, err := net.Interfaces()
if err != nil {
t.Fatalf("net.Interfaces failed: %v", err)
}
for i, ifi := range ift {
if _, ok := isMulticastAvailable(&ifi); !ok {
continue
}
if err := p.JoinGroup(&ifi, gaddr); err != nil {
t.Fatalf("ipv6.PacketConn.JoinGroup %v on %v failed: %v", gaddr, ifi, err)
}
mift = append(mift, &ift[i])
}
for _, ifi := range mift {
if err := p.LeaveGroup(ifi, gaddr); err != nil {
t.Fatalf("ipv6.PacketConn.LeaveGroup %v on %v failed: %v", gaddr, ifi, err)
}
}
}
}
func TestUDPMultiplePacketConnWithMultipleGroupListeners(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
for _, gaddr := range udpMultipleGroupListenerTests {
c1, err := net.ListenPacket("udp6", "[ff02::]:1024") // wildcard address with reusable port
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c1.Close()
c2, err := net.ListenPacket("udp6", "[ff02::]:1024") // wildcard address with reusable port
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c2.Close()
var ps [2]*ipv6.PacketConn
ps[0] = ipv6.NewPacketConn(c1)
ps[1] = ipv6.NewPacketConn(c2)
var mift []*net.Interface
ift, err := net.Interfaces()
if err != nil {
t.Fatalf("net.Interfaces failed: %v", err)
}
for i, ifi := range ift {
if _, ok := isMulticastAvailable(&ifi); !ok {
continue
}
for _, p := range ps {
if err := p.JoinGroup(&ifi, gaddr); err != nil {
t.Fatalf("ipv6.PacketConn.JoinGroup %v on %v failed: %v", gaddr, ifi, err)
}
}
mift = append(mift, &ift[i])
}
for _, ifi := range mift {
for _, p := range ps {
if err := p.LeaveGroup(ifi, gaddr); err != nil {
t.Fatalf("ipv6.PacketConn.LeaveGroup %v on %v failed: %v", gaddr, ifi, err)
}
}
}
}
}
func TestUDPPerInterfaceSinglePacketConnWithSingleGroupListener(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
gaddr := net.IPAddr{IP: net.ParseIP("ff02::114")} // see RFC 4727
type ml struct {
c *ipv6.PacketConn
ifi *net.Interface
}
var mlt []*ml
ift, err := net.Interfaces()
if err != nil {
t.Fatalf("net.Interfaces failed: %v", err)
}
for i, ifi := range ift {
ip, ok := isMulticastAvailable(&ifi)
if !ok {
continue
}
c, err := net.ListenPacket("udp6", fmt.Sprintf("[%s%%%s]:1024", ip.String(), ifi.Name)) // unicast address with non-reusable port
if err != nil {
t.Fatalf("net.ListenPacket with %v failed: %v", ip, err)
}
defer c.Close()
p := ipv6.NewPacketConn(c)
if err := p.JoinGroup(&ifi, &gaddr); err != nil {
t.Fatalf("ipv6.PacketConn.JoinGroup on %v failed: %v", ifi, err)
}
mlt = append(mlt, &ml{p, &ift[i]})
}
for _, m := range mlt {
if err := m.c.LeaveGroup(m.ifi, &gaddr); err != nil {
t.Fatalf("ipv6.PacketConn.LeaveGroup on %v failed: %v", m.ifi, err)
}
}
}
func TestIPSinglePacketConnWithSingleGroupListener(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
if os.Getuid() != 0 {
t.Skip("must be root")
}
c, err := net.ListenPacket("ip6:ipv6-icmp", "::") // wildcard address
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c.Close()
p := ipv6.NewPacketConn(c)
gaddr := net.IPAddr{IP: net.ParseIP("ff02::114")} // see RFC 4727
var mift []*net.Interface
ift, err := net.Interfaces()
if err != nil {
t.Fatalf("net.Interfaces failed: %v", err)
}
for i, ifi := range ift {
if _, ok := isMulticastAvailable(&ifi); !ok {
continue
}
if err := p.JoinGroup(&ifi, &gaddr); err != nil {
t.Fatalf("ipv6.PacketConn.JoinGroup on %v failed: %v", ifi, err)
}
mift = append(mift, &ift[i])
}
for _, ifi := range mift {
if err := p.LeaveGroup(ifi, &gaddr); err != nil {
t.Fatalf("ipv6.PacketConn.LeaveGroup on %v failed: %v", ifi, err)
}
}
}
func TestIPPerInterfaceSinglePacketConnWithSingleGroupListener(t *testing.T) {
switch runtime.GOOS {
case "darwin", "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
if os.Getuid() != 0 {
t.Skip("must be root")
}
gaddr := net.IPAddr{IP: net.ParseIP("ff02::114")} // see RFC 4727
type ml struct {
c *ipv6.PacketConn
ifi *net.Interface
}
var mlt []*ml
ift, err := net.Interfaces()
if err != nil {
t.Fatalf("net.Interfaces failed: %v", err)
}
for i, ifi := range ift {
ip, ok := isMulticastAvailable(&ifi)
if !ok {
continue
}
c, err := net.ListenPacket("ip6:ipv6-icmp", fmt.Sprintf("%s%%%s", ip.String(), ifi.Name)) // unicast address
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c.Close()
p := ipv6.NewPacketConn(c)
if err := p.JoinGroup(&ifi, &gaddr); err != nil {
t.Fatalf("ipv6.PacketConn.JoinGroup on %v failed: %v", ifi, err)
}
mlt = append(mlt, &ml{p, &ift[i]})
}
for _, m := range mlt {
if err := m.c.LeaveGroup(m.ifi, &gaddr); err != nil {
t.Fatalf("ipv6.PacketConn.LeaveGroup on %v failed: %v", m.ifi, err)
}
}
}
@@ -1,76 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6_test
import (
"code.google.com/p/go.net/ipv6"
"net"
"os"
"runtime"
"testing"
)
var packetConnMulticastSocketOptionTests = []struct {
net, proto, addr string
gaddr net.Addr
}{
{"udp6", "", "[ff02::]:0", &net.UDPAddr{IP: net.ParseIP("ff02::114")}}, // see RFC 4727
{"ip6", ":ipv6-icmp", "::", &net.IPAddr{IP: net.ParseIP("ff02::114")}}, // see RFC 4727
}
func TestPacketConnMulticastSocketOptions(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
ifi := loopbackInterface()
if ifi == nil {
t.Skipf("not available on %q", runtime.GOOS)
}
for _, tt := range packetConnMulticastSocketOptionTests {
if tt.net == "ip6" && os.Getuid() != 0 {
t.Skip("must be root")
}
c, err := net.ListenPacket(tt.net+tt.proto, tt.addr)
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c.Close()
p := ipv6.NewPacketConn(c)
hoplim := 255
if err := p.SetMulticastHopLimit(hoplim); err != nil {
t.Fatalf("ipv6.PacketConn.SetMulticastHopLimit failed: %v", err)
}
if v, err := p.MulticastHopLimit(); err != nil {
t.Fatalf("ipv6.PacketConn.MulticastHopLimit failed: %v", err)
} else if v != hoplim {
t.Fatalf("got unexpected multicast hop limit %v; expected %v", v, hoplim)
}
for _, toggle := range []bool{true, false} {
if err := p.SetMulticastLoopback(toggle); err != nil {
t.Fatalf("ipv6.PacketConn.SetMulticastLoopback failed: %v", err)
}
if v, err := p.MulticastLoopback(); err != nil {
t.Fatalf("ipv6.PacketConn.MulticastLoopback failed: %v", err)
} else if v != toggle {
t.Fatalf("got unexpected multicast loopback %v; expected %v", v, toggle)
}
}
if err := p.JoinGroup(ifi, tt.gaddr); err != nil {
t.Fatalf("ipv6.PacketConn.JoinGroup(%v, %v) failed: %v", ifi, tt.gaddr, err)
}
if err := p.LeaveGroup(ifi, tt.gaddr); err != nil {
t.Fatalf("ipv6.PacketConn.LeaveGroup(%v, %v) failed: %v", ifi, tt.gaddr, err)
}
}
}
-15
View File
@@ -1,15 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import "net"
// A payloadHandler represents the IPv6 datagram payload handler.
type payloadHandler struct {
net.PacketConn
rawOpt
}
func (c *payloadHandler) ok() bool { return c != nil && c.PacketConn != nil }
-70
View File
@@ -1,70 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build !plan9,!windows
package ipv6
import (
"net"
"syscall"
)
// ReadFrom reads a payload of the received IPv6 datagram, from the
// endpoint c, copying the payload into b. It returns the number of
// bytes copied into b, the control message cm and the source address
// src of the received datagram.
func (c *payloadHandler) ReadFrom(b []byte) (n int, cm *ControlMessage, src net.Addr, err error) {
if !c.ok() {
return 0, nil, nil, syscall.EINVAL
}
oob := newControlMessage(&c.rawOpt)
var oobn int
switch c := c.PacketConn.(type) {
case *net.UDPConn:
if n, oobn, _, src, err = c.ReadMsgUDP(b, oob); err != nil {
return 0, nil, nil, err
}
case *net.IPConn:
if n, oobn, _, src, err = c.ReadMsgIP(b, oob); err != nil {
return 0, nil, nil, err
}
default:
return 0, nil, nil, errInvalidConnType
}
if cm, err = parseControlMessage(oob[:oobn]); err != nil {
return 0, nil, nil, err
}
if cm != nil {
cm.Src = netAddrToIP16(src)
}
return
}
// WriteTo writes a payload of the IPv6 datagram, to the destination
// address dst through the endpoint c, copying the payload from b. It
// returns the number of bytes written. The control message cm allows
// the IPv6 header fields and the datagram path to be specified. The
// cm may be nil if control of the outgoing datagram is not required.
func (c *payloadHandler) WriteTo(b []byte, cm *ControlMessage, dst net.Addr) (n int, err error) {
if !c.ok() {
return 0, syscall.EINVAL
}
oob := marshalControlMessage(cm)
if dst == nil {
return 0, errMissingAddress
}
switch c := c.PacketConn.(type) {
case *net.UDPConn:
n, _, err = c.WriteMsgUDP(b, oob, dst.(*net.UDPAddr))
case *net.IPConn:
n, _, err = c.WriteMsgIP(b, oob, dst.(*net.IPAddr))
default:
return 0, errInvalidConnType
}
if err != nil {
return 0, err
}
return
}
-41
View File
@@ -1,41 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build plan9 windows
package ipv6
import (
"net"
"syscall"
)
// ReadFrom reads a payload of the received IPv6 datagram, from the
// endpoint c, copying the payload into b. It returns the number of
// bytes copied into b, the control message cm and the source address
// src of the received datagram.
func (c *payloadHandler) ReadFrom(b []byte) (n int, cm *ControlMessage, src net.Addr, err error) {
if !c.ok() {
return 0, nil, nil, syscall.EINVAL
}
if n, src, err = c.PacketConn.ReadFrom(b); err != nil {
return 0, nil, nil, err
}
return
}
// WriteTo writes a payload of the IPv6 datagram, to the destination
// address dst through the endpoint c, copying the payload from b. It
// returns the number of bytes written. The control message cm allows
// the IPv6 header fields and the datagram path to be specified. The
// cm may be nil if control of the outgoing datagram is not required.
func (c *payloadHandler) WriteTo(b []byte, cm *ControlMessage, dst net.Addr) (n int, err error) {
if !c.ok() {
return 0, syscall.EINVAL
}
if dst == nil {
return 0, errMissingAddress
}
return c.PacketConn.WriteTo(b, dst)
}
-168
View File
@@ -1,168 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6_test
import (
"bytes"
"code.google.com/p/go.net/ipv6"
"net"
"runtime"
"sync"
"testing"
)
func benchmarkUDPListener() (net.PacketConn, net.Addr, error) {
c, err := net.ListenPacket("udp6", "[::1]:0")
if err != nil {
return nil, nil, err
}
dst, err := net.ResolveUDPAddr("udp6", c.LocalAddr().String())
if err != nil {
c.Close()
return nil, nil, err
}
return c, dst, nil
}
func BenchmarkReadWriteNetUDP(b *testing.B) {
c, dst, err := benchmarkUDPListener()
if err != nil {
b.Fatalf("benchmarkUDPListener failed: %v", err)
}
defer c.Close()
wb, rb := []byte("HELLO-R-U-THERE"), make([]byte, 128)
b.ResetTimer()
for i := 0; i < b.N; i++ {
benchmarkReadWriteNetUDP(b, c, wb, rb, dst)
}
}
func benchmarkReadWriteNetUDP(b *testing.B, c net.PacketConn, wb, rb []byte, dst net.Addr) {
if _, err := c.WriteTo(wb, dst); err != nil {
b.Fatalf("net.PacketConn.WriteTo failed: %v", err)
}
if _, _, err := c.ReadFrom(rb); err != nil {
b.Fatalf("net.PacketConn.ReadFrom failed: %v", err)
}
}
func BenchmarkReadWriteIPv6UDP(b *testing.B) {
c, dst, err := benchmarkUDPListener()
if err != nil {
b.Fatalf("benchmarkUDPListener failed: %v", err)
}
defer c.Close()
p := ipv6.NewPacketConn(c)
cf := ipv6.FlagTrafficClass | ipv6.FlagHopLimit | ipv6.FlagInterface | ipv6.FlagPathMTU
if err := p.SetControlMessage(cf, true); err != nil {
b.Fatalf("ipv6.PacketConn.SetControlMessage failed: %v", err)
}
ifi := loopbackInterface()
wb, rb := []byte("HELLO-R-U-THERE"), make([]byte, 128)
b.ResetTimer()
for i := 0; i < b.N; i++ {
benchmarkReadWriteIPv6UDP(b, p, wb, rb, dst, ifi)
}
}
func benchmarkReadWriteIPv6UDP(b *testing.B, p *ipv6.PacketConn, wb, rb []byte, dst net.Addr, ifi *net.Interface) {
cm := ipv6.ControlMessage{
TrafficClass: DiffServAF11 | CongestionExperienced,
HopLimit: 1,
}
if ifi != nil {
cm.IfIndex = ifi.Index
}
if n, err := p.WriteTo(wb, &cm, dst); err != nil {
b.Fatalf("ipv6.PacketConn.WriteTo failed: %v", err)
} else if n != len(wb) {
b.Fatalf("ipv6.PacketConn.WriteTo failed: short write: %v", n)
}
if _, _, _, err := p.ReadFrom(rb); err != nil {
b.Fatalf("ipv6.PacketConn.ReadFrom failed: %v", err)
}
}
func TestPacketConnConcurrentReadWriteUnicastUDP(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
c, err := net.ListenPacket("udp6", "[::1]:0")
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c.Close()
p := ipv6.NewPacketConn(c)
defer p.Close()
dst, err := net.ResolveUDPAddr("udp6", c.LocalAddr().String())
if err != nil {
t.Fatalf("net.ResolveUDPAddr failed: %v", err)
}
ifi := loopbackInterface()
cf := ipv6.FlagTrafficClass | ipv6.FlagHopLimit | ipv6.FlagSrc | ipv6.FlagDst | ipv6.FlagInterface | ipv6.FlagPathMTU
wb := []byte("HELLO-R-U-THERE")
var wg sync.WaitGroup
reader := func() {
defer wg.Done()
rb := make([]byte, 128)
if n, cm, _, err := p.ReadFrom(rb); err != nil {
t.Errorf("ipv6.PacketConn.ReadFrom failed: %v", err)
return
} else if !bytes.Equal(rb[:n], wb) {
t.Errorf("got %v; expected %v", rb[:n], wb)
return
} else {
t.Logf("rcvd cmsg: %v", cm)
}
}
writer := func(toggle bool) {
defer wg.Done()
cm := ipv6.ControlMessage{
TrafficClass: DiffServAF11 | CongestionExperienced,
Src: net.IPv6loopback,
Dst: net.IPv6loopback,
}
if ifi != nil {
cm.IfIndex = ifi.Index
}
if err := p.SetControlMessage(cf, toggle); err != nil {
t.Errorf("ipv6.PacketConn.SetControlMessage failed: %v", err)
return
}
if n, err := p.WriteTo(wb, &cm, dst); err != nil {
t.Errorf("ipv6.PacketConn.WriteTo failed: %v", err)
return
} else if n != len(wb) {
t.Errorf("ipv6.PacketConn.WriteTo failed: short write: %v", n)
return
}
}
const N = 10
wg.Add(N)
for i := 0; i < N; i++ {
go reader()
}
wg.Add(2 * N)
for i := 0; i < 2*N; i++ {
go writer(i%2 != 0)
}
wg.Add(N)
for i := 0; i < N; i++ {
go reader()
}
wg.Wait()
}
@@ -1,73 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin
package ipv6
import (
"os"
"unsafe"
)
func ipv6ReceiveTrafficClass(fd int) (bool, error) {
return false, errOpNoSupport
}
func setIPv6ReceiveTrafficClass(fd int, v bool) error {
return errOpNoSupport
}
func ipv6ReceiveHopLimit(fd int) (bool, error) {
var v int32
l := sysSockoptLen(4)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockopt2292HopLimit, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return false, os.NewSyscallError("getsockopt", err)
}
return v == 1, nil
}
func setIPv6ReceiveHopLimit(fd int, v bool) error {
vv := int32(boolint(v))
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockopt2292HopLimit, uintptr(unsafe.Pointer(&vv)), 4))
}
func ipv6ReceivePacketInfo(fd int) (bool, error) {
var v int32
l := sysSockoptLen(4)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockopt2292PacketInfo, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return false, os.NewSyscallError("getsockopt", err)
}
return v == 1, nil
}
func setIPv6ReceivePacketInfo(fd int, v bool) error {
vv := int32(boolint(v))
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockopt2292PacketInfo, uintptr(unsafe.Pointer(&vv)), 4))
}
func ipv6PathMTU(fd int) (int, error) {
return 0, errOpNoSupport
}
func ipv6ReceivePathMTU(fd int) (bool, error) {
return false, errOpNoSupport
}
func setIPv6ReceivePathMTU(fd int, v bool) error {
return errOpNoSupport
}
func ipv6ICMPFilter(fd int) (*ICMPFilter, error) {
var v ICMPFilter
l := sysSockoptLen(sysSizeofICMPFilter)
if err := getsockopt(fd, ianaProtocolIPv6ICMP, sysSockoptICMPFilter, uintptr(unsafe.Pointer(&v.sysICMPFilter)), &l); err != nil {
return nil, os.NewSyscallError("getsockopt", err)
}
return &v, nil
}
func setIPv6ICMPFilter(fd int, f *ICMPFilter) error {
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6ICMP, sysSockoptICMPFilter, uintptr(unsafe.Pointer(&f.sysICMPFilter)), sysSizeofICMPFilter))
}
@@ -1,20 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin freebsd netbsd openbsd
package ipv6
import (
"os"
"unsafe"
)
func setIPv6Checksum(fd int, on bool, offset int) error {
if !on {
offset = -1
}
v := int32(offset)
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockoptChecksum, uintptr(unsafe.Pointer(&v)), 4))
}
@@ -1,18 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"os"
"unsafe"
)
func setIPv6Checksum(fd int, on bool, offset int) error {
if !on {
offset = -1
}
v := int32(offset)
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolReserved, sysSockoptChecksum, uintptr(unsafe.Pointer(&v)), 4))
}
@@ -1,124 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin freebsd linux netbsd openbsd
package ipv6
import (
"net"
"os"
"unsafe"
)
func ipv6TrafficClass(fd int) (int, error) {
var v int32
l := sysSockoptLen(4)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockoptTrafficClass, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return 0, os.NewSyscallError("getsockopt", err)
}
return int(v), nil
}
func setIPv6TrafficClass(fd, v int) error {
vv := int32(v)
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockoptTrafficClass, uintptr(unsafe.Pointer(&vv)), 4))
}
func ipv6HopLimit(fd int) (int, error) {
var v int32
l := sysSockoptLen(4)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockoptUnicastHopLimit, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return 0, os.NewSyscallError("getsockopt", err)
}
return int(v), nil
}
func setIPv6HopLimit(fd, v int) error {
vv := int32(v)
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockoptUnicastHopLimit, uintptr(unsafe.Pointer(&vv)), 4))
}
func ipv6Checksum(fd int) (bool, int, error) {
var v int32
l := sysSockoptLen(4)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockoptChecksum, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return false, 0, os.NewSyscallError("getsockopt", err)
}
on := true
if v == -1 {
on = false
}
return on, int(v), nil
}
func ipv6MulticastHopLimit(fd int) (int, error) {
var v int32
l := sysSockoptLen(4)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockoptMulticastHopLimit, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return 0, os.NewSyscallError("getsockopt", err)
}
return int(v), nil
}
func setIPv6MulticastHopLimit(fd, v int) error {
vv := int32(v)
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockoptMulticastHopLimit, uintptr(unsafe.Pointer(&vv)), 4))
}
func ipv6MulticastInterface(fd int) (*net.Interface, error) {
var v int32
l := sysSockoptLen(4)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockoptMulticastInterface, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return nil, os.NewSyscallError("getsockopt", err)
}
if v == 0 {
return nil, nil
}
ifi, err := net.InterfaceByIndex(int(v))
if err != nil {
return nil, err
}
return ifi, nil
}
func setIPv6MulticastInterface(fd int, ifi *net.Interface) error {
var v int32
if ifi != nil {
v = int32(ifi.Index)
}
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockoptMulticastInterface, uintptr(unsafe.Pointer(&v)), 4))
}
func ipv6MulticastLoopback(fd int) (bool, error) {
var v int32
l := sysSockoptLen(4)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockoptMulticastLoopback, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return false, os.NewSyscallError("getsockopt", err)
}
return v == 1, nil
}
func setIPv6MulticastLoopback(fd int, v bool) error {
vv := int32(boolint(v))
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockoptMulticastLoopback, uintptr(unsafe.Pointer(&vv)), 4))
}
func joinIPv6Group(fd int, ifi *net.Interface, grp net.IP) error {
mreq := sysMulticastReq{}
copy(mreq.IP[:], grp)
if ifi != nil {
mreq.IfIndex = uint32(ifi.Index)
}
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockoptJoinGroup, uintptr(unsafe.Pointer(&mreq)), sysSizeofMulticastReq))
}
func leaveIPv6Group(fd int, ifi *net.Interface, grp net.IP) error {
mreq := sysMulticastReq{}
copy(mreq.IP[:], grp)
if ifi != nil {
mreq.IfIndex = uint32(ifi.Index)
}
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockoptLeaveGroup, uintptr(unsafe.Pointer(&mreq)), sysSizeofMulticastReq))
}
@@ -1,116 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"net"
"os"
"syscall"
"unsafe"
)
func ipv6TrafficClass(fd syscall.Handle) (int, error) {
// TODO(mikio): Implement this
return 0, syscall.EWINDOWS
}
func setIPv6TrafficClass(fd syscall.Handle, v int) error {
// TODO(mikio): Implement this
return syscall.EWINDOWS
}
func ipv6HopLimit(fd syscall.Handle) (int, error) {
var v int32
l := int32(4)
if err := syscall.Getsockopt(fd, ianaProtocolIPv6, sysSockoptUnicastHopLimit, (*byte)(unsafe.Pointer(&v)), &l); err != nil {
return 0, os.NewSyscallError("getsockopt", err)
}
return int(v), nil
}
func setIPv6HopLimit(fd syscall.Handle, v int) error {
vv := int32(v)
return os.NewSyscallError("setsockopt", syscall.Setsockopt(fd, ianaProtocolIPv6, sysSockoptUnicastHopLimit, (*byte)(unsafe.Pointer(&vv)), 4))
}
func ipv6Checksum(fd syscall.Handle) (bool, int, error) {
// TODO(mikio): Implement this
return false, 0, syscall.EWINDOWS
}
func ipv6MulticastHopLimit(fd syscall.Handle) (int, error) {
var v int32
l := int32(4)
if err := syscall.Getsockopt(fd, ianaProtocolIPv6, sysSockoptMulticastHopLimit, (*byte)(unsafe.Pointer(&v)), &l); err != nil {
return 0, os.NewSyscallError("getsockopt", err)
}
return int(v), nil
}
func setIPv6MulticastHopLimit(fd syscall.Handle, v int) error {
vv := int32(v)
return os.NewSyscallError("setsockopt", syscall.Setsockopt(fd, ianaProtocolIPv6, sysSockoptMulticastHopLimit, (*byte)(unsafe.Pointer(&vv)), 4))
}
func ipv6MulticastInterface(fd syscall.Handle) (*net.Interface, error) {
var v int32
l := int32(4)
if err := syscall.Getsockopt(fd, ianaProtocolIPv6, sysSockoptMulticastInterface, (*byte)(unsafe.Pointer(&v)), &l); err != nil {
return nil, os.NewSyscallError("getsockopt", err)
}
if v == 0 {
return nil, nil
}
ifi, err := net.InterfaceByIndex(int(v))
if err != nil {
return nil, err
}
return ifi, nil
}
func setIPv6MulticastInterface(fd syscall.Handle, ifi *net.Interface) error {
var v int32
if ifi != nil {
v = int32(ifi.Index)
}
return os.NewSyscallError("setsockopt", syscall.Setsockopt(fd, ianaProtocolIPv6, sysSockoptMulticastInterface, (*byte)(unsafe.Pointer(&v)), 4))
}
func ipv6MulticastLoopback(fd syscall.Handle) (bool, error) {
var v int32
l := int32(4)
if err := syscall.Getsockopt(fd, ianaProtocolIPv6, sysSockoptMulticastLoopback, (*byte)(unsafe.Pointer(&v)), &l); err != nil {
return false, os.NewSyscallError("getsockopt", err)
}
return v == 1, nil
}
func setIPv6MulticastLoopback(fd syscall.Handle, v bool) error {
vv := int32(boolint(v))
return os.NewSyscallError("setsockopt", syscall.Setsockopt(fd, ianaProtocolIPv6, sysSockoptMulticastLoopback, (*byte)(unsafe.Pointer(&vv)), 4))
}
func joinIPv6Group(fd syscall.Handle, ifi *net.Interface, grp net.IP) error {
mreq := sysMulticastReq{}
copy(mreq.IP[:], grp)
if ifi != nil {
mreq.IfIndex = uint32(ifi.Index)
}
return os.NewSyscallError("setsockopt", syscall.Setsockopt(fd, ianaProtocolIPv6, sysSockoptJoinGroup, (*byte)(unsafe.Pointer(&mreq)), int32(sysSizeofMulticastReq)))
}
func leaveIPv6Group(fd syscall.Handle, ifi *net.Interface, grp net.IP) error {
mreq := sysMulticastReq{}
copy(mreq.IP[:], grp)
if ifi != nil {
mreq.IfIndex = uint32(ifi.Index)
}
return os.NewSyscallError("setsockopt", syscall.Setsockopt(fd, ianaProtocolIPv6, sysSockoptLeaveGroup, (*byte)(unsafe.Pointer(&mreq)), int32(sysSizeofMulticastReq)))
}
func setIPv6Checksum(fd syscall.Handle, on bool, offset int) error {
// TODO(mikio): Implement this
return syscall.EWINDOWS
}
@@ -1,12 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build dragonfly plan9 solaris
package ipv6
func ipv6PathMTU(fd int) (int, error) {
// TODO(mikio): Implement this
return 0, errOpNoSupport
}
@@ -1,90 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build freebsd linux netbsd openbsd
package ipv6
import (
"os"
"unsafe"
)
func ipv6ReceiveTrafficClass(fd int) (bool, error) {
var v int32
l := sysSockoptLen(4)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockoptReceiveTrafficClass, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return false, os.NewSyscallError("getsockopt", err)
}
return v == 1, nil
}
func setIPv6ReceiveTrafficClass(fd int, v bool) error {
vv := int32(boolint(v))
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockoptReceiveTrafficClass, uintptr(unsafe.Pointer(&vv)), 4))
}
func ipv6ReceiveHopLimit(fd int) (bool, error) {
var v int32
l := sysSockoptLen(4)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockoptReceiveHopLimit, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return false, os.NewSyscallError("getsockopt", err)
}
return v == 1, nil
}
func setIPv6ReceiveHopLimit(fd int, v bool) error {
vv := int32(boolint(v))
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockoptReceiveHopLimit, uintptr(unsafe.Pointer(&vv)), 4))
}
func ipv6ReceivePacketInfo(fd int) (bool, error) {
var v int32
l := sysSockoptLen(4)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockoptReceivePacketInfo, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return false, os.NewSyscallError("getsockopt", err)
}
return v == 1, nil
}
func setIPv6ReceivePacketInfo(fd int, v bool) error {
vv := int32(boolint(v))
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockoptReceivePacketInfo, uintptr(unsafe.Pointer(&vv)), 4))
}
func ipv6PathMTU(fd int) (int, error) {
var v sysMTUInfo
l := sysSockoptLen(sysSizeofMTUInfo)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockoptPathMTU, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return 0, os.NewSyscallError("getsockopt", err)
}
return int(v.MTU), nil
}
func ipv6ReceivePathMTU(fd int) (bool, error) {
var v int32
l := sysSockoptLen(4)
if err := getsockopt(fd, ianaProtocolIPv6, sysSockoptReceivePathMTU, uintptr(unsafe.Pointer(&v)), &l); err != nil {
return false, os.NewSyscallError("getsockopt", err)
}
return v == 1, nil
}
func setIPv6ReceivePathMTU(fd int, v bool) error {
vv := int32(boolint(v))
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6, sysSockoptReceivePathMTU, uintptr(unsafe.Pointer(&vv)), 4))
}
func ipv6ICMPFilter(fd int) (*ICMPFilter, error) {
var v ICMPFilter
l := sysSockoptLen(sysSizeofICMPFilter)
if err := getsockopt(fd, ianaProtocolIPv6ICMP, sysSockoptICMPFilter, uintptr(unsafe.Pointer(&v.sysICMPFilter)), &l); err != nil {
return nil, os.NewSyscallError("getsockopt", err)
}
return &v, nil
}
func setIPv6ICMPFilter(fd int, f *ICMPFilter) error {
return os.NewSyscallError("setsockopt", setsockopt(fd, ianaProtocolIPv6ICMP, sysSockoptICMPFilter, uintptr(unsafe.Pointer(&f.sysICMPFilter)), sysSizeofICMPFilter))
}
@@ -1,62 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import "syscall"
func ipv6ReceiveTrafficClass(fd syscall.Handle) (bool, error) {
// TODO(mikio): Implement this
return false, syscall.EWINDOWS
}
func setIPv6ReceiveTrafficClass(fd syscall.Handle, v bool) error {
// TODO(mikio): Implement this
return syscall.EWINDOWS
}
func ipv6ReceiveHopLimit(fd syscall.Handle) (bool, error) {
// TODO(mikio): Implement this
return false, syscall.EWINDOWS
}
func setIPv6ReceiveHopLimit(fd syscall.Handle, v bool) error {
// TODO(mikio): Implement this
return syscall.EWINDOWS
}
func ipv6ReceivePacketInfo(fd syscall.Handle) (bool, error) {
// TODO(mikio): Implement this
return false, syscall.EWINDOWS
}
func setIPv6ReceivePacketInfo(fd syscall.Handle, v bool) error {
// TODO(mikio): Implement this
return syscall.EWINDOWS
}
func ipv6PathMTU(fd syscall.Handle) (int, error) {
// TODO(mikio): Implement this
return 0, syscall.EWINDOWS
}
func ipv6ReceivePathMTU(fd syscall.Handle) (bool, error) {
// TODO(mikio): Implement this
return false, syscall.EWINDOWS
}
func setIPv6ReceivePathMTU(fd syscall.Handle, v bool) error {
// TODO(mikio): Implement this
return syscall.EWINDOWS
}
func ipv6ICMPFilter(fd syscall.Handle) (*ICMPFilter, error) {
// TODO(mikio): Implement this
return nil, syscall.EWINDOWS
}
func setIPv6ICMPFilter(fd syscall.Handle, f *ICMPFilter) error {
// TODO(mikio): Implement this
return syscall.EWINDOWS
}
-136
View File
@@ -1,136 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6_test
import (
"code.google.com/p/go.net/ipv6"
"net"
"os"
"runtime"
"testing"
)
var supportsIPv6 bool
func init() {
if ln, err := net.Listen("tcp6", "[::1]:0"); err == nil {
ln.Close()
supportsIPv6 = true
}
}
var condFatalf = func() func(*testing.T, string, ...interface{}) {
// A few APIs are not implemented yet on some platforms.
switch runtime.GOOS {
case "darwin", "dragonfly", "plan9", "solaris", "windows":
return (*testing.T).Logf
}
return (*testing.T).Fatalf
}()
func TestConnInitiatorPathMTU(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
ln, err := net.Listen("tcp6", "[::1]:0")
if err != nil {
t.Fatalf("net.Listen failed: %v", err)
}
defer ln.Close()
done := make(chan bool)
go acceptor(t, ln, done)
c, err := net.Dial("tcp6", ln.Addr().String())
if err != nil {
t.Fatalf("net.Dial failed: %v", err)
}
defer c.Close()
if pmtu, err := ipv6.NewConn(c).PathMTU(); err != nil {
condFatalf(t, "ipv6.Conn.PathMTU failed: %v", err)
} else {
t.Logf("path mtu for %v: %v", c.RemoteAddr(), pmtu)
}
<-done
}
func TestConnResponderPathMTU(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
ln, err := net.Listen("tcp6", "[::1]:0")
if err != nil {
t.Fatalf("net.Listen failed: %v", err)
}
defer ln.Close()
done := make(chan bool)
go connector(t, "tcp6", ln.Addr().String(), done)
c, err := ln.Accept()
if err != nil {
t.Fatalf("net.Accept failed: %v", err)
}
defer c.Close()
if pmtu, err := ipv6.NewConn(c).PathMTU(); err != nil {
condFatalf(t, "ipv6.Conn.PathMTU failed: %v", err)
} else {
t.Logf("path mtu for %v: %v", c.RemoteAddr(), pmtu)
}
<-done
}
func TestPacketConnChecksum(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
if os.Getuid() != 0 {
t.Skip("must be root")
}
c, err := net.ListenPacket("ip6:89", "::") // OSPF for IPv6
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c.Close()
p := ipv6.NewPacketConn(c)
offset := 12 // see RFC 5340
for _, toggle := range []bool{false, true} {
if err := p.SetChecksum(toggle, offset); err != nil {
if toggle {
t.Fatalf("ipv6.PacketConn.SetChecksum(%v, %v) failed: %v", toggle, offset, err)
} else {
// Some platforms never allow to disable the kernel
// checksum processing.
t.Logf("ipv6.PacketConn.SetChecksum(%v, %v) failed: %v", toggle, offset, err)
}
}
if on, offset, err := p.Checksum(); err != nil {
t.Fatalf("ipv6.PacketConn.Checksum failed: %v", err)
} else {
t.Logf("kernel checksum processing enabled=%v, offset=%v", on, offset)
}
}
}
-23
View File
@@ -1,23 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
type sysSockoptLen uint32
const (
sysSizeofPacketInfo = 0x14
sysSizeofMulticastReq = 0x14
sysSizeofICMPFilter = 0x20
)
type sysPacketInfo struct {
IP [16]byte
IfIndex uint32
}
type sysMulticastReq struct {
IP [16]byte
IfIndex uint32
}
-48
View File
@@ -1,48 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build freebsd netbsd openbsd
package ipv6
import (
"net"
"syscall"
)
// RFC 3493 options
const (
// See /usr/include/netinet6/in6.h.
sysSockoptUnicastHopLimit = 0x4
sysSockoptMulticastHopLimit = 0xa
sysSockoptMulticastInterface = 0x9
sysSockoptMulticastLoopback = 0xb
sysSockoptJoinGroup = 0xc
sysSockoptLeaveGroup = 0xd
)
// RFC 3542 options
const (
// See /usr/include/netinet6/in6.h.
sysSockoptReceiveTrafficClass = 0x39
sysSockoptTrafficClass = 0x3d
sysSockoptReceiveHopLimit = 0x25
sysSockoptHopLimit = 0x2f
sysSockoptReceivePacketInfo = 0x24
sysSockoptPacketInfo = 0x2e
sysSockoptReceivePathMTU = 0x2b
sysSockoptPathMTU = 0x2c
sysSockoptNextHop = 0x30
sysSockoptChecksum = 0x1a
// See /usr/include/netinet6/in6.h.
sysSockoptICMPFilter = 0x12
)
func setSockaddr(sa *syscall.RawSockaddrInet6, ip net.IP, ifindex int) {
sa.Len = syscall.SizeofSockaddrInet6
sa.Family = syscall.AF_INET6
copy(sa.Addr[:], ip)
sa.Scope_id = uint32(ifindex)
}
-54
View File
@@ -1,54 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"net"
"syscall"
)
// RFC 2292 options
const (
// See /usr/include/netinet6/in6.h.
sysSockopt2292HopLimit = 0x14
sysSockopt2292PacketInfo = 0x13
sysSockopt2292NextHop = 0x15
)
// RFC 3493 options
const (
// See /usr/include/netinet6/in6.h.
sysSockoptUnicastHopLimit = 0x4
sysSockoptMulticastHopLimit = 0xa
sysSockoptMulticastInterface = 0x9
sysSockoptMulticastLoopback = 0xb
sysSockoptJoinGroup = 0xc
sysSockoptLeaveGroup = 0xd
)
// RFC 3542 options
const (
// See /usr/include/netinet6/in6.h.
sysSockoptReceiveTrafficClass = 0x23
sysSockoptTrafficClass = 0x24
sysSockoptReceiveHopLimit = 0x25
sysSockoptHopLimit = 0x2f
sysSockoptReceivePacketInfo = 0x3d
sysSockoptPacketInfo = 0x2e
sysSockoptReceivePathMTU = 0x2b
sysSockoptPathMTU = 0x2c
sysSockoptNextHop = 0x30
sysSockoptChecksum = 0x1a
// See /usr/include/netinet6/in6.h.
sysSockoptICMPFilter = 0x12
)
func setSockaddr(sa *syscall.RawSockaddrInet6, ip net.IP, ifindex int) {
sa.Len = syscall.SizeofSockaddrInet6
sa.Family = syscall.AF_INET6
copy(sa.Addr[:], ip)
sa.Scope_id = uint32(ifindex)
}
-45
View File
@@ -1,45 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"net"
"syscall"
)
// RFC 3493 options
const (
// See /usr/include/linux/in6.h.
sysSockoptUnicastHopLimit = 0x10
sysSockoptMulticastHopLimit = 0x12
sysSockoptMulticastInterface = 0x11
sysSockoptMulticastLoopback = 0x13
sysSockoptJoinGroup = 0x14
sysSockoptLeaveGroup = 0x15
)
// RFC 3542 options
const (
// See /usr/include/linux/ipv6.h,in6.h.
sysSockoptReceiveTrafficClass = 0x42
sysSockoptTrafficClass = 0x43
sysSockoptReceiveHopLimit = 0x33
sysSockoptHopLimit = 0x34
sysSockoptReceivePacketInfo = 0x31
sysSockoptPacketInfo = 0x32
sysSockoptReceivePathMTU = 0x3c
sysSockoptPathMTU = 0x3d
sysSockoptNextHop = 0x9
sysSockoptChecksum = 0x7
// See /usr/include/linux/icmpv6.h.
sysSockoptICMPFilter = 0x1
)
func setSockaddr(sa *syscall.RawSockaddrInet6, ip net.IP, ifindex int) {
sa.Family = syscall.AF_INET6
copy(sa.Addr[:], ip)
sa.Scope_id = uint32(ifindex)
}
-16
View File
@@ -1,16 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin freebsd linux netbsd openbsd
package ipv6
import "syscall"
const sysSizeofMTUInfo = 0x20
type sysMTUInfo struct {
Addr syscall.RawSockaddrInet6
MTU uint32
}
-33
View File
@@ -1,33 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6
import (
"net"
"syscall"
)
// RFC 3493 options
const (
// See ws2tcpip.h.
sysSockoptUnicastHopLimit = 0x4
sysSockoptMulticastHopLimit = 0xa
sysSockoptMulticastInterface = 0x9
sysSockoptMulticastLoopback = 0xb
sysSockoptJoinGroup = 0xc
sysSockoptLeaveGroup = 0xd
)
// RFC 3542 options
const (
// See ws2tcpip.h.
sysSockoptPacketInfo = 0x13
)
func setSockaddr(sa *syscall.RawSockaddrInet6, ip net.IP, ifindex int) {
sa.Family = syscall.AF_INET6
copy(sa.Addr[:], ip)
sa.Scope_id = uint32(ifindex)
}
@@ -1,42 +0,0 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// This code is a duplicate of syscall/syscall_linux_386.go with small
// modifications.
package ipv6
import (
"syscall"
"unsafe"
)
// On x86 Linux, all the socket calls go through an extra indirection,
// I think because the 5-register system call interface can't handle
// the 6-argument calls like sendto and recvfrom. Instead the
// arguments to the underlying system call are the number below and a
// pointer to an array of uintptr. We hide the pointer in the
// socketcall assembly to avoid allocation on every system call.
const (
// See /usr/include/linux/net.h.
_SETSOCKOPT = 14
_GETSOCKOPT = 15
)
var socketcall func(call int, a0, a1, a2, a3, a4, a5 uintptr) (int, syscall.Errno)
func getsockopt(fd int, level int, name int, v uintptr, l *sysSockoptLen) error {
if _, errno := socketcall(_GETSOCKOPT, uintptr(fd), uintptr(level), uintptr(name), uintptr(v), uintptr(unsafe.Pointer(l)), 0); errno != 0 {
return error(errno)
}
return nil
}
func setsockopt(fd int, level int, name int, v uintptr, l uintptr) error {
if _, errno := socketcall(_SETSOCKOPT, uintptr(fd), uintptr(level), uintptr(name), v, l, 0); errno != 0 {
return error(errno)
}
return nil
}
@@ -1,56 +0,0 @@
// Copyright 2009 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// This code is a duplicate of syscall/syscall_linux_386.s with small
// modifications.
#define SYS_SOCKETCALL 102 // from zsysnum_linux_386.go
// func socketcallnosplit7(call int, a0, a1, a2, a3, a4, a5 uintptr) (n int, errno int)
// Kernel interface gets call sub-number and pointer to a0 for Go 1.1.
TEXT ·socketcallnosplit7(SB),7,$0
CALL runtime·entersyscall(SB)
MOVL $SYS_SOCKETCALL, AX // syscall entry
MOVL 4(SP), BX // socket call number
LEAL 8(SP), CX // pointer to call arguments
MOVL $0, DX
MOVL $0, SI
MOVL $0, DI
CALL *runtime·_vdso(SB)
CMPL AX, $0xfffff001
JLS ok1
MOVL $-1, 32(SP) // n
NEGL AX
MOVL AX, 36(SP) // errno
CALL runtime·exitsyscall(SB)
RET
ok1:
MOVL AX, 32(SP) // n
MOVL $0, 36(SP) // errno
CALL runtime·exitsyscall(SB)
RET
// func socketcallnosplit4(call int, a0, a1, a2, a3, a4, a5 uintptr) (n int, errno int)
// Kernel interface gets call sub-number and pointer to a0 for Go 1.2.
TEXT ·socketcallnosplit4(SB),4,$0-40
CALL runtime·entersyscall(SB)
MOVL $SYS_SOCKETCALL, AX // syscall entry
MOVL 4(SP), BX // socket call number
LEAL 8(SP), CX // pointer to call arguments
MOVL $0, DX
MOVL $0, SI
MOVL $0, DI
CALL *runtime·_vdso(SB)
CMPL AX, $0xfffff001
JLS ok2
MOVL $-1, 32(SP) // n
NEGL AX
MOVL AX, 36(SP) // errno
CALL runtime·exitsyscall(SB)
RET
ok2:
MOVL AX, 32(SP) // n
MOVL $0, 36(SP) // errno
CALL runtime·exitsyscall(SB)
RET
@@ -1,15 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.2
package ipv6
import "syscall"
func socketcallnosplit4(call int, a0, a1, a2, a3, a4, a5 uintptr) (int, syscall.Errno)
func init() {
socketcall = socketcallnosplit4
}
@@ -1,15 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build go1.1,!go1.2
package ipv6
import "syscall"
func socketcallnosplit7(call int, a0, a1, a2, a3, a4, a5 uintptr) (int, syscall.Errno)
func init() {
socketcall = socketcallnosplit7
}
-26
View File
@@ -1,26 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin freebsd linux,amd64 linux,arm netbsd openbsd
package ipv6
import (
"syscall"
"unsafe"
)
func getsockopt(fd int, level, name int, v uintptr, l *sysSockoptLen) error {
if _, _, errno := syscall.Syscall6(syscall.SYS_GETSOCKOPT, uintptr(fd), uintptr(level), uintptr(name), uintptr(v), uintptr(unsafe.Pointer(l)), 0); errno != 0 {
return error(errno)
}
return nil
}
func setsockopt(fd int, level int, name int, v uintptr, l uintptr) error {
if _, _, errno := syscall.Syscall6(syscall.SYS_SETSOCKOPT, uintptr(fd), uintptr(level), uintptr(name), uintptr(v), uintptr(l), 0); errno != 0 {
return error(errno)
}
return nil
}
-172
View File
@@ -1,172 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6_test
import (
"bytes"
"code.google.com/p/go.net/ipv6"
"net"
"os"
"runtime"
"testing"
"time"
)
func TestPacketConnReadWriteUnicastUDP(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
c, err := net.ListenPacket("udp6", "[::1]:0")
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c.Close()
p := ipv6.NewPacketConn(c)
defer p.Close()
dst, err := net.ResolveUDPAddr("udp6", c.LocalAddr().String())
if err != nil {
t.Fatalf("net.ResolveUDPAddr failed: %v", err)
}
cm := ipv6.ControlMessage{
TrafficClass: DiffServAF11 | CongestionExperienced,
Src: net.IPv6loopback,
Dst: net.IPv6loopback,
}
cf := ipv6.FlagTrafficClass | ipv6.FlagHopLimit | ipv6.FlagSrc | ipv6.FlagDst | ipv6.FlagInterface | ipv6.FlagPathMTU
ifi := loopbackInterface()
if ifi != nil {
cm.IfIndex = ifi.Index
}
wb := []byte("HELLO-R-U-THERE")
for i, toggle := range []bool{true, false, true} {
if err := p.SetControlMessage(cf, toggle); err != nil {
t.Fatalf("ipv6.PacketConn.SetControlMessage failed: %v", err)
}
cm.HopLimit = i + 1
if err := p.SetWriteDeadline(time.Now().Add(100 * time.Millisecond)); err != nil {
t.Fatalf("ipv6.PacketConn.SetWriteDeadline failed: %v", err)
}
if n, err := p.WriteTo(wb, &cm, dst); err != nil {
t.Fatalf("ipv6.PacketConn.WriteTo failed: %v", err)
} else if n != len(wb) {
t.Fatalf("ipv6.PacketConn.WriteTo failed: short write: %v", n)
}
rb := make([]byte, 128)
if err := p.SetReadDeadline(time.Now().Add(100 * time.Millisecond)); err != nil {
t.Fatalf("ipv6.PacketConn.SetReadDeadline failed: %v", err)
}
if n, cm, _, err := p.ReadFrom(rb); err != nil {
t.Fatalf("ipv6.PacketConn.ReadFrom failed: %v", err)
} else if !bytes.Equal(rb[:n], wb) {
t.Fatalf("got %v; expected %v", rb[:n], wb)
} else {
t.Logf("rcvd cmsg: %v", cm)
}
}
}
func TestPacketConnReadWriteUnicastICMP(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
if os.Getuid() != 0 {
t.Skip("must be root")
}
c, err := net.ListenPacket("ip6:ipv6-icmp", "::1")
if err != nil {
t.Fatalf("net.ListenPacket failed: %v", err)
}
defer c.Close()
p := ipv6.NewPacketConn(c)
defer p.Close()
dst, err := net.ResolveIPAddr("ip6", "::1")
if err != nil {
t.Fatalf("net.ResolveIPAddr failed: %v", err)
}
pshicmp := ipv6PseudoHeader(c.LocalAddr().(*net.IPAddr).IP, dst.IP, ianaProtocolIPv6ICMP)
cm := ipv6.ControlMessage{
TrafficClass: DiffServAF11 | CongestionExperienced,
Src: net.IPv6loopback,
Dst: net.IPv6loopback,
}
cf := ipv6.FlagTrafficClass | ipv6.FlagHopLimit | ipv6.FlagSrc | ipv6.FlagDst | ipv6.FlagInterface | ipv6.FlagPathMTU
ifi := loopbackInterface()
if ifi != nil {
cm.IfIndex = ifi.Index
}
var f ipv6.ICMPFilter
f.SetAll(true)
f.Set(ipv6.ICMPTypeEchoReply, false)
if err := p.SetICMPFilter(&f); err != nil {
t.Fatalf("ipv6.PacketConn.SetICMPFilter failed: %v", err)
}
var psh []byte
for i, toggle := range []bool{true, false, true} {
if toggle {
psh = nil
if err := p.SetChecksum(true, 2); err != nil {
t.Fatalf("ipv6.PacketConn.SetChecksum failed: %v", err)
}
} else {
psh = pshicmp
// Some platforms never allow to disable the
// kernel checksum processing.
p.SetChecksum(false, -1)
}
wb, err := (&icmpMessage{
Type: ipv6.ICMPTypeEchoRequest, Code: 0,
Body: &icmpEcho{
ID: os.Getpid() & 0xffff, Seq: i + 1,
Data: []byte("HELLO-R-U-THERE"),
},
}).Marshal(psh)
if err != nil {
t.Fatalf("icmpMessage.Marshal failed: %v", err)
}
if err := p.SetControlMessage(cf, toggle); err != nil {
t.Fatalf("ipv6.PacketConn.SetControlMessage failed: %v", err)
}
cm.HopLimit = i + 1
if err := p.SetWriteDeadline(time.Now().Add(100 * time.Millisecond)); err != nil {
t.Fatalf("ipv6.PacketConn.SetWriteDeadline failed: %v", err)
}
if n, err := p.WriteTo(wb, &cm, dst); err != nil {
t.Fatalf("ipv6.PacketConn.WriteTo failed: %v", err)
} else if n != len(wb) {
t.Fatalf("ipv6.PacketConn.WriteTo failed: short write: %v", n)
}
rb := make([]byte, 128)
if err := p.SetReadDeadline(time.Now().Add(100 * time.Millisecond)); err != nil {
t.Fatalf("ipv6.PacketConn.SetReadDeadline failed: %v", err)
}
if n, cm, _, err := p.ReadFrom(rb); err != nil {
t.Fatalf("ipv6.PacketConn.ReadFrom failed: %v", err)
} else {
t.Logf("rcvd cmsg: %v", cm)
if m, err := parseICMPMessage(rb[:n]); err != nil {
t.Fatalf("parseICMPMessage failed: %v", err)
} else if m.Type != ipv6.ICMPTypeEchoReply || m.Code != 0 {
t.Fatalf("got type=%v, code=%v; expected type=%v, code=%v", m.Type, m.Code, ipv6.ICMPTypeEchoReply, 0)
}
}
}
}
@@ -1,101 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package ipv6_test
import (
"code.google.com/p/go.net/ipv6"
"net"
"os"
"runtime"
"testing"
)
func TestConnUnicastSocketOptions(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
ln, err := net.Listen("tcp6", "[::1]:0")
if err != nil {
t.Fatalf("net.Listen failed: %v", err)
}
defer ln.Close()
done := make(chan bool)
go acceptor(t, ln, done)
c, err := net.Dial("tcp6", ln.Addr().String())
if err != nil {
t.Fatalf("net.Dial failed: %v", err)
}
defer c.Close()
testUnicastSocketOptions(t, ipv6.NewConn(c))
<-done
}
var packetConnUnicastSocketOptionTests = []struct {
net, proto, addr string
}{
{"udp6", "", "[::1]:0"},
{"ip6", ":ipv6-icmp", "::1"},
}
func TestPacketConnUnicastSocketOptions(t *testing.T) {
switch runtime.GOOS {
case "dragonfly", "plan9", "solaris", "windows":
t.Skipf("not supported on %q", runtime.GOOS)
}
if !supportsIPv6 {
t.Skip("ipv6 is not supported")
}
for _, tt := range packetConnUnicastSocketOptionTests {
if tt.net == "ip6" && os.Getuid() != 0 {
t.Skip("must be root")
}
c, err := net.ListenPacket(tt.net+tt.proto, tt.addr)
if err != nil {
t.Fatalf("net.ListenPacket(%q, %q) failed: %v", tt.net+tt.proto, tt.addr, err)
}
defer c.Close()
testUnicastSocketOptions(t, ipv6.NewPacketConn(c))
}
}
type testIPv6UnicastConn interface {
TrafficClass() (int, error)
SetTrafficClass(int) error
HopLimit() (int, error)
SetHopLimit(int) error
}
func testUnicastSocketOptions(t *testing.T, c testIPv6UnicastConn) {
tclass := DiffServCS0 | NotECNTransport
if err := c.SetTrafficClass(tclass); err != nil {
t.Fatalf("ipv6.Conn.SetTrafficClass failed: %v", err)
}
if v, err := c.TrafficClass(); err != nil {
t.Fatalf("ipv6.Conn.TrafficClass failed: %v", err)
} else if v != tclass {
t.Fatalf("got unexpected traffic class %v; expected %v", v, tclass)
}
hoplim := 255
if err := c.SetHopLimit(hoplim); err != nil {
t.Fatalf("ipv6.Conn.SetHopLimit failed: %v", err)
}
if v, err := c.HopLimit(); err != nil {
t.Fatalf("ipv6.Conn.HopLimit failed: %v", err)
} else if v != hoplim {
t.Fatalf("got unexpected hop limit %v; expected %v", v, hoplim)
}
}
@@ -1,37 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package transform_test
import (
"fmt"
"unicode"
"code.google.com/p/go.text/transform"
"code.google.com/p/go.text/unicode/norm"
)
func ExampleRemoveFunc() {
input := []byte(`tschüß; до свидания`)
b := make([]byte, len(input))
t := transform.RemoveFunc(unicode.IsSpace)
n, _, _ := t.Transform(b, input, true)
fmt.Println(string(b[:n]))
t = transform.RemoveFunc(func(r rune) bool {
return !unicode.Is(unicode.Latin, r)
})
n, _, _ = t.Transform(b, input, true)
fmt.Println(string(b[:n]))
n, _, _ = t.Transform(b, norm.NFD.Bytes(input), true)
fmt.Println(string(b[:n]))
// Output:
// tschüß;досвидания
// tschüß
// tschuß
}
-496
View File
@@ -1,496 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package transform provides reader and writer wrappers that transform the
// bytes passing through as well as various transformations. Example
// transformations provided by other packages include normalization and
// conversion between character sets.
package transform
import (
"errors"
"io"
"unicode/utf8"
)
var (
// ErrShortDst means that the destination buffer was too short to
// receive all of the transformed bytes.
ErrShortDst = errors.New("transform: short destination buffer")
// ErrShortSrc means that the source buffer has insufficient data to
// complete the transformation.
ErrShortSrc = errors.New("transform: short source buffer")
// errInconsistentByteCount means that Transform returned success (nil
// error) but also returned nSrc inconsistent with the src argument.
errInconsistentByteCount = errors.New("transform: inconsistent byte count returned")
// errShortInternal means that an internal buffer is not large enough
// to make progress and the Transform operation must be aborted.
errShortInternal = errors.New("transform: short internal buffer")
)
// Transformer transforms bytes.
type Transformer interface {
// Transform writes to dst the transformed bytes read from src, and
// returns the number of dst bytes written and src bytes read. The
// atEOF argument tells whether src represents the last bytes of the
// input.
//
// Callers should always process the nDst bytes produced and account
// for the nSrc bytes consumed before considering the error err.
//
// A nil error means that all of the transformed bytes (whether freshly
// transformed from src or left over from previous Transform calls)
// were written to dst. A nil error can be returned regardless of
// whether atEOF is true. If err is nil then nSrc must equal len(src);
// the converse is not necessarily true.
//
// ErrShortDst means that dst was too short to receive all of the
// transformed bytes. ErrShortSrc means that src had insufficient data
// to complete the transformation. If both conditions apply, then
// either error may be returned. Other than the error conditions listed
// here, implementations are free to report other errors that arise.
Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error)
}
// TODO: Do we require that a Transformer be reusable if it returns a nil error
// or do we always require a reset after use? Is Reset mandatory or optional?
// Reader wraps another io.Reader by transforming the bytes read.
type Reader struct {
r io.Reader
t Transformer
err error
// dst[dst0:dst1] contains bytes that have been transformed by t but
// not yet copied out via Read.
dst []byte
dst0, dst1 int
// src[src0:src1] contains bytes that have been read from r but not
// yet transformed through t.
src []byte
src0, src1 int
// transformComplete is whether the transformation is complete,
// regardless of whether or not it was successful.
transformComplete bool
}
const defaultBufSize = 4096
// NewReader returns a new Reader that wraps r by transforming the bytes read
// via t.
func NewReader(r io.Reader, t Transformer) *Reader {
return &Reader{
r: r,
t: t,
dst: make([]byte, defaultBufSize),
src: make([]byte, defaultBufSize),
}
}
// Read implements the io.Reader interface.
func (r *Reader) Read(p []byte) (int, error) {
n, err := 0, error(nil)
for {
// Copy out any transformed bytes and return the final error if we are done.
if r.dst0 != r.dst1 {
n = copy(p, r.dst[r.dst0:r.dst1])
r.dst0 += n
if r.dst0 == r.dst1 && r.transformComplete {
return n, r.err
}
return n, nil
} else if r.transformComplete {
return 0, r.err
}
// Try to transform some source bytes, or to flush the transformer if we
// are out of source bytes. We do this even if r.r.Read returned an error.
// As the io.Reader documentation says, "process the n > 0 bytes returned
// before considering the error".
if r.src0 != r.src1 || r.err != nil {
r.dst0 = 0
r.dst1, n, err = r.t.Transform(r.dst, r.src[r.src0:r.src1], r.err == io.EOF)
r.src0 += n
switch {
case err == nil:
if r.src0 != r.src1 {
r.err = errInconsistentByteCount
}
// The Transform call was successful; we are complete if we
// cannot read more bytes into src.
r.transformComplete = r.err != nil
continue
case err == ErrShortDst && r.dst1 != 0:
// Make room in dst by copying out, and try again.
continue
case err == ErrShortSrc && r.src1-r.src0 != len(r.src) && r.err == nil:
// Read more bytes into src via the code below, and try again.
default:
r.transformComplete = true
// The reader error (r.err) takes precedence over the
// transformer error (err) unless r.err is nil or io.EOF.
if r.err == nil || r.err == io.EOF {
r.err = err
}
continue
}
}
// Move any untransformed source bytes to the start of the buffer
// and read more bytes.
if r.src0 != 0 {
r.src0, r.src1 = 0, copy(r.src, r.src[r.src0:r.src1])
}
n, r.err = r.r.Read(r.src[r.src1:])
r.src1 += n
}
}
// TODO: implement ReadByte (and ReadRune??).
// Writer wraps another io.Writer by transforming the bytes read.
// The user needs to call Close to flush unwritten bytes that may
// be buffered.
type Writer struct {
w io.Writer
t Transformer
dst []byte
// src[:n] contains bytes that have not yet passed through t.
src []byte
n int
}
// NewWriter returns a new Writer that wraps w by transforming the bytes written
// via t.
func NewWriter(w io.Writer, t Transformer) *Writer {
return &Writer{
w: w,
t: t,
dst: make([]byte, defaultBufSize),
src: make([]byte, defaultBufSize),
}
}
// Write implements the io.Writer interface. If there are not enough
// bytes available to complete a Transform, the bytes will be buffered
// for the next write. Call Close to convert the remaining bytes.
func (w *Writer) Write(data []byte) (n int, err error) {
src := data
if w.n > 0 {
// Append bytes from data to the last remainder.
// TODO: limit the amount copied on first try.
n = copy(w.src[w.n:], data)
w.n += n
src = w.src[:w.n]
}
for {
nDst, nSrc, err := w.t.Transform(w.dst, src, false)
if _, werr := w.w.Write(w.dst[:nDst]); werr != nil {
return n, werr
}
src = src[nSrc:]
if w.n > 0 && len(src) <= n {
// Enough bytes from w.src have been consumed. We make src point
// to data instead to reduce the copying.
w.n = 0
n -= len(src)
src = data[n:]
if n < len(data) && (err == nil || err == ErrShortSrc) {
continue
}
} else {
n += nSrc
}
switch {
case err == ErrShortDst && nDst > 0:
case err == ErrShortSrc && len(src) < len(w.src):
m := copy(w.src, src)
// If w.n > 0, bytes from data were already copied to w.src and n
// was already set to the number of bytes consumed.
if w.n == 0 {
n += m
}
w.n = m
return n, nil
case err == nil && w.n > 0:
return n, errInconsistentByteCount
default:
return n, err
}
}
}
// Close implements the io.Closer interface.
func (w *Writer) Close() error {
for src := w.src[:w.n]; len(src) > 0; {
nDst, nSrc, err := w.t.Transform(w.dst, src, true)
if nDst == 0 {
return err
}
if _, werr := w.w.Write(w.dst[:nDst]); werr != nil {
return werr
}
if err != ErrShortDst {
return err
}
src = src[nSrc:]
}
return nil
}
type nop struct{}
func (nop) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
n := copy(dst, src)
if n < len(src) {
err = ErrShortDst
}
return n, n, err
}
type discard struct{}
func (discard) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
return 0, len(src), nil
}
var (
// Discard is a Transformer for which all Transform calls succeed
// by consuming all bytes and writing nothing.
Discard Transformer = discard{}
// Nop is a Transformer that copies src to dst.
Nop Transformer = nop{}
)
// chain is a sequence of links. A chain with N Transformers has N+1 links and
// N+1 buffers. Of those N+1 buffers, the first and last are the src and dst
// buffers given to chain.Transform and the middle N-1 buffers are intermediate
// buffers owned by the chain. The i'th link transforms bytes from the i'th
// buffer chain.link[i].b at read offset chain.link[i].p to the i+1'th buffer
// chain.link[i+1].b at write offset chain.link[i+1].n, for i in [0, N).
type chain struct {
link []link
err error
// errStart is the index at which the error occurred plus 1. Processing
// errStart at this level at the next call to Transform. As long as
// errStart > 0, chain will not consume any more source bytes.
errStart int
}
func (c *chain) fatalError(errIndex int, err error) {
if i := errIndex + 1; i > c.errStart {
c.errStart = i
c.err = err
}
}
type link struct {
t Transformer
// b[p:n] holds the bytes to be transformed by t.
b []byte
p int
n int
}
func (l *link) src() []byte {
return l.b[l.p:l.n]
}
func (l *link) dst() []byte {
return l.b[l.n:]
}
// Chain returns a Transformer that applies t in sequence.
func Chain(t ...Transformer) Transformer {
if len(t) == 0 {
return nop{}
}
c := &chain{link: make([]link, len(t)+1)}
for i, tt := range t {
c.link[i].t = tt
}
// Allocate intermediate buffers.
b := make([][defaultBufSize]byte, len(t)-1)
for i := range b {
c.link[i+1].b = b[i][:]
}
return c
}
// Transform applies the transformers of c in sequence.
func (c *chain) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
// Set up src and dst in the chain.
srcL := &c.link[0]
dstL := &c.link[len(c.link)-1]
srcL.b, srcL.p, srcL.n = src, 0, len(src)
dstL.b, dstL.n = dst, 0
var lastFull, needProgress bool // for detecting progress
// i is the index of the next Transformer to apply, for i in [low, high].
// low is the lowest index for which c.link[low] may still produce bytes.
// high is the highest index for which c.link[high] has a Transformer.
// The error returned by Transform determines whether to increase or
// decrease i. We try to completely fill a buffer before converting it.
for low, i, high := c.errStart, c.errStart, len(c.link)-2; low <= i && i <= high; {
in, out := &c.link[i], &c.link[i+1]
nDst, nSrc, err0 := in.t.Transform(out.dst(), in.src(), atEOF && low == i)
out.n += nDst
in.p += nSrc
if i > 0 && in.p == in.n {
in.p, in.n = 0, 0
}
needProgress, lastFull = lastFull, false
switch err0 {
case ErrShortDst:
// Process the destination buffer next. Return if we are already
// at the high index.
if i == high {
return dstL.n, srcL.p, ErrShortDst
}
if out.n != 0 {
i++
// If the Transformer at the next index is not able to process any
// source bytes there is nothing that can be done to make progress
// and the bytes will remain unprocessed. lastFull is used to
// detect this and break out of the loop with a fatal error.
lastFull = true
continue
}
// The destination buffer was too small, but is completely empty.
// Return a fatal error as this transformation can never complete.
c.fatalError(i, errShortInternal)
case ErrShortSrc:
if i == 0 {
// Save ErrShortSrc in err. All other errors take precedence.
err = ErrShortSrc
break
}
// Source bytes were depleted before filling up the destination buffer.
// Verify we made some progress, move the remaining bytes to the errStart
// and try to get more source bytes.
if needProgress && nSrc == 0 || in.n-in.p == len(in.b) {
// There were not enough source bytes to proceed while the source
// buffer cannot hold any more bytes. Return a fatal error as this
// transformation can never complete.
c.fatalError(i, errShortInternal)
break
}
// in.b is an internal buffer and we can make progress.
in.p, in.n = 0, copy(in.b, in.src())
fallthrough
case nil:
// if i == low, we have depleted the bytes at index i or any lower levels.
// In that case we increase low and i. In all other cases we decrease i to
// fetch more bytes before proceeding to the next index.
if i > low {
i--
continue
}
default:
c.fatalError(i, err0)
}
// Exhausted level low or fatal error: increase low and continue
// to process the bytes accepted so far.
i++
low = i
}
// If c.errStart > 0, this means we found a fatal error. We will clear
// all upstream buffers. At this point, no more progress can be made
// downstream, as Transform would have bailed while handling ErrShortDst.
if c.errStart > 0 {
for i := 1; i < c.errStart; i++ {
c.link[i].p, c.link[i].n = 0, 0
}
err, c.errStart, c.err = c.err, 0, nil
}
return dstL.n, srcL.p, err
}
// RemoveFunc returns a Transformer that removes from the input all runes r for
// which f(r) is true. Illegal bytes in the input are replaced by RuneError.
func RemoveFunc(f func(r rune) bool) Transformer {
return removeF(f)
}
type removeF func(r rune) bool
// Transform implements the Transformer interface.
func (t removeF) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
for r, sz := rune(0), 0; len(src) > 0; src = src[sz:] {
if r = rune(src[0]); r < utf8.RuneSelf {
sz = 1
} else {
r, sz = utf8.DecodeRune(src)
if sz == 1 {
// Invalid rune.
if !atEOF && !utf8.FullRune(src[nSrc:]) {
err = ErrShortSrc
break
}
// We replace illegal bytes with RuneError. Not doing so might
// otherwise turn a sequence of invalid UTF-8 into valid UTF-8.
// The resulting byte sequence may subsequently contain runes
// for which t(r) is true that were passed unnoticed.
if !t(r) {
if nDst+3 > len(dst) {
err = ErrShortDst
break
}
nDst += copy(dst[nDst:], "\uFFFD")
}
nSrc++
continue
}
}
if !t(r) {
if nDst+sz > len(dst) {
err = ErrShortDst
break
}
nDst += copy(dst[nDst:], src[:sz])
}
nSrc += sz
}
return
}
// Bytes returns a new byte slice with the result of converting b using t.
// If any unrecoverable error occurs it returns nil.
func Bytes(t Transformer, b []byte) []byte {
out := make([]byte, len(b))
n := 0
for {
nDst, nSrc, err := t.Transform(out[n:], b, true)
n += nDst
if err == nil {
return out[:n]
} else if err != ErrShortDst {
return nil
}
b = b[nSrc:]
// Grow the destination buffer.
sz := len(out)
if sz <= 256 {
sz *= 2
} else {
sz += sz >> 1
}
out2 := make([]byte, sz)
copy(out2, out[:n])
out = out2
}
}
@@ -1,901 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package transform
import (
"bytes"
"errors"
"fmt"
"io/ioutil"
"strconv"
"strings"
"testing"
"unicode/utf8"
)
type lowerCaseASCII struct{}
func (lowerCaseASCII) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
n := len(src)
if n > len(dst) {
n, err = len(dst), ErrShortDst
}
for i, c := range src[:n] {
if 'A' <= c && c <= 'Z' {
c += 'a' - 'A'
}
dst[i] = c
}
return n, n, err
}
var errYouMentionedX = errors.New("you mentioned X")
type dontMentionX struct{}
func (dontMentionX) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
n := len(src)
if n > len(dst) {
n, err = len(dst), ErrShortDst
}
for i, c := range src[:n] {
if c == 'X' {
return i, i, errYouMentionedX
}
dst[i] = c
}
return n, n, err
}
// doublerAtEOF is a strange Transformer that transforms "this" to "tthhiiss",
// but only if atEOF is true.
type doublerAtEOF struct{}
func (doublerAtEOF) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
if !atEOF {
return 0, 0, ErrShortSrc
}
for i, c := range src {
if 2*i+2 >= len(dst) {
return 2 * i, i, ErrShortDst
}
dst[2*i+0] = c
dst[2*i+1] = c
}
return 2 * len(src), len(src), nil
}
// rleDecode and rleEncode implement a toy run-length encoding: "aabbbbbbbbbb"
// is encoded as "2a10b". The decoding is assumed to not contain any numbers.
type rleDecode struct{}
func (rleDecode) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
loop:
for len(src) > 0 {
n := 0
for i, c := range src {
if '0' <= c && c <= '9' {
n = 10*n + int(c-'0')
continue
}
if i == 0 {
return nDst, nSrc, errors.New("rleDecode: bad input")
}
if n > len(dst) {
return nDst, nSrc, ErrShortDst
}
for j := 0; j < n; j++ {
dst[j] = c
}
dst, src = dst[n:], src[i+1:]
nDst, nSrc = nDst+n, nSrc+i+1
continue loop
}
if atEOF {
return nDst, nSrc, errors.New("rleDecode: bad input")
}
return nDst, nSrc, ErrShortSrc
}
return nDst, nSrc, nil
}
type rleEncode struct {
// allowStutter means that "xxxxxxxx" can be encoded as "5x3x"
// instead of always as "8x".
allowStutter bool
}
func (e rleEncode) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
for len(src) > 0 {
n, c0 := len(src), src[0]
for i, c := range src[1:] {
if c != c0 {
n = i + 1
break
}
}
if n == len(src) && !atEOF && !e.allowStutter {
return nDst, nSrc, ErrShortSrc
}
s := strconv.Itoa(n)
if len(s) >= len(dst) {
return nDst, nSrc, ErrShortDst
}
copy(dst, s)
dst[len(s)] = c0
dst, src = dst[len(s)+1:], src[n:]
nDst, nSrc = nDst+len(s)+1, nSrc+n
}
return nDst, nSrc, nil
}
type testCase struct {
desc string
t Transformer
src string
dstSize int
srcSize int
ioSize int
wantStr string
wantErr error
wantIter int // number of iterations taken; 0 means we don't care.
}
func (t testCase) String() string {
return tstr(t.t) + "; " + t.desc
}
func tstr(t Transformer) string {
if stringer, ok := t.(fmt.Stringer); ok {
return stringer.String()
}
s := fmt.Sprintf("%T", t)
return s[1+strings.Index(s, "."):]
}
func (c chain) String() string {
buf := &bytes.Buffer{}
buf.WriteString("Chain(")
for i, l := range c.link[:len(c.link)-1] {
if i != 0 {
fmt.Fprint(buf, ", ")
}
buf.WriteString(tstr(l.t))
}
buf.WriteString(")")
return buf.String()
}
var testCases = []testCase{
{
desc: "basic",
t: lowerCaseASCII{},
src: "Hello WORLD.",
dstSize: 100,
srcSize: 100,
wantStr: "hello world.",
},
{
desc: "small dst",
t: lowerCaseASCII{},
src: "Hello WORLD.",
dstSize: 3,
srcSize: 100,
wantStr: "hello world.",
},
{
desc: "small src",
t: lowerCaseASCII{},
src: "Hello WORLD.",
dstSize: 100,
srcSize: 4,
wantStr: "hello world.",
},
{
desc: "small buffers",
t: lowerCaseASCII{},
src: "Hello WORLD.",
dstSize: 3,
srcSize: 4,
wantStr: "hello world.",
},
{
desc: "very small buffers",
t: lowerCaseASCII{},
src: "Hello WORLD.",
dstSize: 1,
srcSize: 1,
wantStr: "hello world.",
},
{
desc: "basic",
t: dontMentionX{},
src: "The First Rule of Transform Club: don't mention Mister X, ever.",
dstSize: 100,
srcSize: 100,
wantStr: "The First Rule of Transform Club: don't mention Mister ",
wantErr: errYouMentionedX,
},
{
desc: "small buffers",
t: dontMentionX{},
src: "The First Rule of Transform Club: don't mention Mister X, ever.",
dstSize: 10,
srcSize: 10,
wantStr: "The First Rule of Transform Club: don't mention Mister ",
wantErr: errYouMentionedX,
},
{
desc: "very small buffers",
t: dontMentionX{},
src: "The First Rule of Transform Club: don't mention Mister X, ever.",
dstSize: 1,
srcSize: 1,
wantStr: "The First Rule of Transform Club: don't mention Mister ",
wantErr: errYouMentionedX,
},
{
desc: "only transform at EOF",
t: doublerAtEOF{},
src: "this",
dstSize: 100,
srcSize: 100,
wantStr: "tthhiiss",
},
{
desc: "basic",
t: rleDecode{},
src: "1a2b3c10d11e0f1g",
dstSize: 100,
srcSize: 100,
wantStr: "abbcccddddddddddeeeeeeeeeeeg",
},
{
desc: "long",
t: rleDecode{},
src: "12a23b34c45d56e99z",
dstSize: 100,
srcSize: 100,
wantStr: strings.Repeat("a", 12) +
strings.Repeat("b", 23) +
strings.Repeat("c", 34) +
strings.Repeat("d", 45) +
strings.Repeat("e", 56) +
strings.Repeat("z", 99),
},
{
desc: "tight buffers",
t: rleDecode{},
src: "1a2b3c10d11e0f1g",
dstSize: 11,
srcSize: 3,
wantStr: "abbcccddddddddddeeeeeeeeeeeg",
},
{
desc: "short dst",
t: rleDecode{},
src: "1a2b3c10d11e0f1g",
dstSize: 10,
srcSize: 3,
wantStr: "abbcccdddddddddd",
wantErr: ErrShortDst,
},
{
desc: "short src",
t: rleDecode{},
src: "1a2b3c10d11e0f1g",
dstSize: 11,
srcSize: 2,
ioSize: 2,
wantStr: "abbccc",
wantErr: ErrShortSrc,
},
{
desc: "basic",
t: rleEncode{},
src: "abbcccddddddddddeeeeeeeeeeeg",
dstSize: 100,
srcSize: 100,
wantStr: "1a2b3c10d11e1g",
},
{
desc: "long",
t: rleEncode{},
src: strings.Repeat("a", 12) +
strings.Repeat("b", 23) +
strings.Repeat("c", 34) +
strings.Repeat("d", 45) +
strings.Repeat("e", 56) +
strings.Repeat("z", 99),
dstSize: 100,
srcSize: 100,
wantStr: "12a23b34c45d56e99z",
},
{
desc: "tight buffers",
t: rleEncode{},
src: "abbcccddddddddddeeeeeeeeeeeg",
dstSize: 3,
srcSize: 12,
wantStr: "1a2b3c10d11e1g",
},
{
desc: "short dst",
t: rleEncode{},
src: "abbcccddddddddddeeeeeeeeeeeg",
dstSize: 2,
srcSize: 12,
wantStr: "1a2b3c",
wantErr: ErrShortDst,
},
{
desc: "short src",
t: rleEncode{},
src: "abbcccddddddddddeeeeeeeeeeeg",
dstSize: 3,
srcSize: 11,
ioSize: 11,
wantStr: "1a2b3c10d",
wantErr: ErrShortSrc,
},
{
desc: "allowStutter = false",
t: rleEncode{allowStutter: false},
src: "aaaabbbbbbbbccccddddd",
dstSize: 10,
srcSize: 10,
wantStr: "4a8b4c5d",
},
{
desc: "allowStutter = true",
t: rleEncode{allowStutter: true},
src: "aaaabbbbbbbbccccddddd",
dstSize: 10,
srcSize: 10,
ioSize: 10,
wantStr: "4a6b2b4c4d1d",
},
}
func TestReader(t *testing.T) {
for _, tc := range testCases {
reset(tc.t)
r := NewReader(strings.NewReader(tc.src), tc.t)
// Differently sized dst and src buffers are not part of the
// exported API. We override them manually.
r.dst = make([]byte, tc.dstSize)
r.src = make([]byte, tc.srcSize)
got, err := ioutil.ReadAll(r)
str := string(got)
if str != tc.wantStr || err != tc.wantErr {
t.Errorf("%s:\ngot %q, %v\nwant %q, %v", tc, str, err, tc.wantStr, tc.wantErr)
}
}
}
func reset(t Transformer) {
var dst [128]byte
for err := ErrShortDst; err != nil; {
_, _, err = t.Transform(dst[:], nil, true)
}
}
func TestWriter(t *testing.T) {
tests := append(testCases, chainTests()...)
for _, tc := range tests {
sizes := []int{1, 2, 3, 4, 5, 10, 100, 1000}
if tc.ioSize > 0 {
sizes = []int{tc.ioSize}
}
for _, sz := range sizes {
bb := &bytes.Buffer{}
reset(tc.t)
w := NewWriter(bb, tc.t)
// Differently sized dst and src buffers are not part of the
// exported API. We override them manually.
w.dst = make([]byte, tc.dstSize)
w.src = make([]byte, tc.srcSize)
src := make([]byte, sz)
var err error
for b := tc.src; len(b) > 0 && err == nil; {
n := copy(src, b)
b = b[n:]
m := 0
m, err = w.Write(src[:n])
if m != n && err == nil {
t.Errorf("%s:%d: did not consume all bytes %d < %d", tc, sz, m, n)
}
}
if err == nil {
err = w.Close()
}
str := bb.String()
if str != tc.wantStr || err != tc.wantErr {
t.Errorf("%s:%d:\ngot %q, %v\nwant %q, %v", tc, sz, str, err, tc.wantStr, tc.wantErr)
}
}
}
}
func TestNop(t *testing.T) {
testCases := []struct {
str string
dstSize int
err error
}{
{"", 0, nil},
{"", 10, nil},
{"a", 0, ErrShortDst},
{"a", 1, nil},
{"a", 10, nil},
}
for i, tc := range testCases {
dst := make([]byte, tc.dstSize)
nDst, nSrc, err := Nop.Transform(dst, []byte(tc.str), true)
want := tc.str
if tc.dstSize < len(want) {
want = want[:tc.dstSize]
}
if got := string(dst[:nDst]); got != want || err != tc.err || nSrc != nDst {
t.Errorf("%d:\ngot %q, %d, %v\nwant %q, %d, %v", i, got, nSrc, err, want, nDst, tc.err)
}
}
}
func TestDiscard(t *testing.T) {
testCases := []struct {
str string
dstSize int
}{
{"", 0},
{"", 10},
{"a", 0},
{"ab", 10},
}
for i, tc := range testCases {
nDst, nSrc, err := Discard.Transform(make([]byte, tc.dstSize), []byte(tc.str), true)
if nDst != 0 || nSrc != len(tc.str) || err != nil {
t.Errorf("%d:\ngot %q, %d, %v\nwant 0, %d, nil", i, nDst, nSrc, err, len(tc.str))
}
}
}
// mkChain creates a Chain transformer. x must be alternating between transformer
// and bufSize, like T, (sz, T)*
func mkChain(x ...interface{}) *chain {
t := []Transformer{}
for i := 0; i < len(x); i += 2 {
t = append(t, x[i].(Transformer))
}
c := Chain(t...).(*chain)
for i, j := 1, 1; i < len(x); i, j = i+2, j+1 {
c.link[j].b = make([]byte, x[i].(int))
}
return c
}
func chainTests() []testCase {
return []testCase{
{
desc: "nil error",
t: mkChain(rleEncode{}, 100, lowerCaseASCII{}),
src: "ABB",
dstSize: 100,
srcSize: 100,
wantStr: "1a2b",
wantErr: nil,
wantIter: 1,
},
{
desc: "short dst buffer",
t: mkChain(lowerCaseASCII{}, 3, rleDecode{}),
src: "1a2b3c10d11e0f1g",
dstSize: 10,
srcSize: 3,
wantStr: "abbcccdddddddddd",
wantErr: ErrShortDst,
},
{
desc: "short internal dst buffer",
t: mkChain(lowerCaseASCII{}, 3, rleDecode{}, 10, Nop),
src: "1a2b3c10d11e0f1g",
dstSize: 100,
srcSize: 3,
wantStr: "abbcccdddddddddd",
wantErr: errShortInternal,
},
{
desc: "short internal dst buffer from input",
t: mkChain(rleDecode{}, 10, Nop),
src: "1a2b3c10d11e0f1g",
dstSize: 100,
srcSize: 3,
wantStr: "abbcccdddddddddd",
wantErr: errShortInternal,
},
{
desc: "empty short internal dst buffer",
t: mkChain(lowerCaseASCII{}, 3, rleDecode{}, 10, Nop),
src: "4a7b11e0f1g",
dstSize: 100,
srcSize: 3,
wantStr: "aaaabbbbbbb",
wantErr: errShortInternal,
},
{
desc: "empty short internal dst buffer from input",
t: mkChain(rleDecode{}, 10, Nop),
src: "4a7b11e0f1g",
dstSize: 100,
srcSize: 3,
wantStr: "aaaabbbbbbb",
wantErr: errShortInternal,
},
{
desc: "short internal src buffer after full dst buffer",
t: mkChain(Nop, 5, rleEncode{}, 10, Nop),
src: "cccccddddd",
dstSize: 100,
srcSize: 100,
wantStr: "",
wantErr: errShortInternal,
wantIter: 1,
},
{
desc: "short internal src buffer after short dst buffer; test lastFull",
t: mkChain(rleDecode{}, 5, rleEncode{}, 4, Nop),
src: "2a1b4c6d",
dstSize: 100,
srcSize: 100,
wantStr: "2a1b",
wantErr: errShortInternal,
},
{
desc: "short internal src buffer after successful complete fill",
t: mkChain(Nop, 3, rleDecode{}),
src: "123a4b",
dstSize: 4,
srcSize: 3,
wantStr: "",
wantErr: errShortInternal,
wantIter: 1,
},
{
desc: "short internal src buffer after short dst buffer; test lastFull",
t: mkChain(rleDecode{}, 5, rleEncode{}),
src: "2a1b4c6d",
dstSize: 4,
srcSize: 100,
wantStr: "2a1b",
wantErr: errShortInternal,
},
{
desc: "short src buffer",
t: mkChain(rleEncode{}, 5, Nop),
src: "abbcccddddeeeee",
dstSize: 4,
srcSize: 4,
ioSize: 4,
wantStr: "1a2b3c",
wantErr: ErrShortSrc,
},
{
desc: "process all in one go",
t: mkChain(rleEncode{}, 5, Nop),
src: "abbcccddddeeeeeffffff",
dstSize: 100,
srcSize: 100,
wantStr: "1a2b3c4d5e6f",
wantErr: nil,
wantIter: 1,
},
{
desc: "complete processing downstream after error",
t: mkChain(dontMentionX{}, 2, rleDecode{}, 5, Nop),
src: "3a4b5eX",
dstSize: 100,
srcSize: 100,
ioSize: 100,
wantStr: "aaabbbbeeeee",
wantErr: errYouMentionedX,
},
{
desc: "return downstream fatal errors first (followed by short dst)",
t: mkChain(dontMentionX{}, 8, rleDecode{}, 4, Nop),
src: "3a4b5eX",
dstSize: 100,
srcSize: 100,
ioSize: 100,
wantStr: "aaabbbb",
wantErr: errShortInternal,
},
{
desc: "return downstream fatal errors first (followed by short src)",
t: mkChain(dontMentionX{}, 5, Nop, 1, rleDecode{}),
src: "1a5bX",
dstSize: 100,
srcSize: 100,
ioSize: 100,
wantStr: "",
wantErr: errShortInternal,
},
{
desc: "short internal",
t: mkChain(Nop, 11, rleEncode{}, 3, Nop),
src: "abbcccddddddddddeeeeeeeeeeeg",
dstSize: 3,
srcSize: 100,
wantStr: "1a2b3c10d",
wantErr: errShortInternal,
},
}
}
func doTransform(tc testCase) (res string, iter int, err error) {
reset(tc.t)
dst := make([]byte, tc.dstSize)
out, in := make([]byte, 0, 2*len(tc.src)), []byte(tc.src)
for {
iter++
src, atEOF := in, true
if len(src) > tc.srcSize {
src, atEOF = src[:tc.srcSize], false
}
nDst, nSrc, err := tc.t.Transform(dst, src, atEOF)
out = append(out, dst[:nDst]...)
in = in[nSrc:]
switch {
case err == nil && len(in) != 0:
case err == ErrShortSrc && nSrc > 0:
case err == ErrShortDst && nDst > 0:
default:
return string(out), iter, err
}
}
}
func TestChain(t *testing.T) {
if c, ok := Chain().(nop); !ok {
t.Errorf("empty chain: %v; want Nop", c)
}
// Test Chain for a single Transformer.
for _, tc := range testCases {
tc.t = Chain(tc.t)
str, _, err := doTransform(tc)
if str != tc.wantStr || err != tc.wantErr {
t.Errorf("%s:\ngot %q, %v\nwant %q, %v", tc, str, err, tc.wantStr, tc.wantErr)
}
}
tests := chainTests()
sizes := []int{1, 2, 3, 4, 5, 7, 10, 100, 1000}
addTest := func(tc testCase, t *chain) {
if t.link[0].t != tc.t && tc.wantErr == ErrShortSrc {
tc.wantErr = errShortInternal
}
if t.link[len(t.link)-2].t != tc.t && tc.wantErr == ErrShortDst {
tc.wantErr = errShortInternal
}
tc.t = t
tests = append(tests, tc)
}
for _, tc := range testCases {
for _, sz := range sizes {
tt := tc
tt.dstSize = sz
addTest(tt, mkChain(tc.t, tc.dstSize, Nop))
addTest(tt, mkChain(tc.t, tc.dstSize, Nop, 2, Nop))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Nop))
if sz >= tc.dstSize && (tc.wantErr != ErrShortDst || sz == tc.dstSize) {
addTest(tt, mkChain(Nop, tc.srcSize, tc.t))
addTest(tt, mkChain(Nop, 100, Nop, tc.srcSize, tc.t))
}
}
}
for _, tc := range testCases {
tt := tc
tt.dstSize = 1
tt.wantStr = ""
addTest(tt, mkChain(tc.t, tc.dstSize, Discard))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Discard))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Nop, tc.dstSize, Discard))
}
for _, tc := range testCases {
tt := tc
tt.dstSize = 100
tt.wantStr = strings.Replace(tc.src, "0f", "", -1)
// Chain encoders and decoders.
if _, ok := tc.t.(rleEncode); ok && tc.wantErr == nil {
addTest(tt, mkChain(tc.t, tc.dstSize, Nop, 1000, rleDecode{}))
addTest(tt, mkChain(tc.t, tc.dstSize, Nop, tc.dstSize, rleDecode{}))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Nop, 100, rleDecode{}))
// decoding needs larger destinations
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, rleDecode{}, 100, Nop))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Nop, 100, rleDecode{}, 100, Nop))
} else if _, ok := tc.t.(rleDecode); ok && tc.wantErr == nil {
// The internal buffer size may need to be the sum of the maximum segment
// size of the two encoders!
addTest(tt, mkChain(tc.t, 2*tc.dstSize, rleEncode{}))
addTest(tt, mkChain(tc.t, tc.dstSize, Nop, 101, rleEncode{}))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Nop, 100, rleEncode{}))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Nop, 200, rleEncode{}, 100, Nop))
}
}
for _, tc := range tests {
str, iter, err := doTransform(tc)
mi := tc.wantIter != 0 && tc.wantIter != iter
if str != tc.wantStr || err != tc.wantErr || mi {
t.Errorf("%s:\ngot iter:%d, %q, %v\nwant iter:%d, %q, %v", tc, iter, str, err, tc.wantIter, tc.wantStr, tc.wantErr)
}
break
}
}
func TestRemoveFunc(t *testing.T) {
filter := RemoveFunc(func(r rune) bool {
return strings.IndexRune("ab\u0300\u1234,", r) != -1
})
tests := []testCase{
{
src: ",",
wantStr: "",
},
{
src: "c",
wantStr: "c",
},
{
src: "\u2345",
wantStr: "\u2345",
},
{
src: "tschüß",
wantStr: "tschüß",
},
{
src: ",до,свидания,",
wantStr: "досвидания",
},
{
src: "a\xbd\xb2=\xbc ⌘",
wantStr: "\uFFFD\uFFFD=\uFFFD ⌘",
},
{
// If we didn't replace illegal bytes with RuneError, the result
// would be \u0300 or the code would need to be more complex.
src: "\xcc\u0300\x80",
wantStr: "\uFFFD\uFFFD",
},
{
src: "\xcc\u0300\x80",
dstSize: 3,
wantStr: "\uFFFD\uFFFD",
wantIter: 2,
},
{
src: "\u2345",
dstSize: 2,
wantStr: "",
wantErr: ErrShortDst,
},
{
src: "\xcc",
dstSize: 2,
wantStr: "",
wantErr: ErrShortDst,
},
{
src: "\u0300",
dstSize: 2,
srcSize: 1,
wantStr: "",
wantErr: ErrShortSrc,
},
{
t: RemoveFunc(func(r rune) bool {
return r == utf8.RuneError
}),
src: "\xcc\u0300\x80",
wantStr: "\u0300",
},
}
for _, tc := range tests {
tc.desc = tc.src
if tc.t == nil {
tc.t = filter
}
if tc.dstSize == 0 {
tc.dstSize = 100
}
if tc.srcSize == 0 {
tc.srcSize = 100
}
str, iter, err := doTransform(tc)
mi := tc.wantIter != 0 && tc.wantIter != iter
if str != tc.wantStr || err != tc.wantErr || mi {
t.Errorf("%+q:\ngot iter:%d, %+q, %v\nwant iter:%d, %+q, %v", tc.src, iter, str, err, tc.wantIter, tc.wantStr, tc.wantErr)
}
tc.src = str
idem, _, _ := doTransform(tc)
if str != idem {
t.Errorf("%+q: found %+q; want %+q", tc.src, idem, str)
}
}
}
func TestBytes(t *testing.T) {
for _, tt := range append(testCases, chainTests()...) {
if tt.desc == "allowStutter = true" {
// We don't have control over the buffer size, so we eliminate tests
// that depend on a specific buffer size being set.
continue
}
got := Bytes(tt.t, []byte(tt.src))
if tt.wantErr != nil {
if tt.wantErr != ErrShortDst && tt.wantErr != ErrShortSrc {
// Bytes should return nil for non-recoverable errors.
if g, w := (got == nil), (tt.wantErr != nil); g != w {
t.Errorf("%s:error: got %v; want %v", tt.desc, g, w)
}
}
// The output strings in the tests that expect an error will
// almost certainly not be the same as the result of Bytes.
continue
}
if string(got) != tt.wantStr {
t.Errorf("%s:string: got %q; want %q", tt.desc, got, tt.wantStr)
}
}
}
-30
View File
@@ -1,30 +0,0 @@
# Copyright 2011 The Go Authors. All rights reserved.
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.
maketables: maketables.go triegen.go
go build $^
maketesttables: maketesttables.go triegen.go
go build $^
normregtest: normregtest.go
go build $^
tables: maketables
./maketables > tables.go
gofmt -w tables.go
trietesttables: maketesttables
./maketesttables > triedata_test.go
gofmt -w triedata_test.go
# Downloads from www.unicode.org, so not part
# of standard test scripts.
test: testtables regtest
testtables: maketables
./maketables -test > data_test.go && go test -tags=test
regtest: normregtest
./normregtest
@@ -1,514 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import "unicode/utf8"
const (
maxNonStarters = 30
// The maximum number of characters needed for a buffer is
// maxNonStarters + 1 for the starter + 1 for the GCJ
maxBufferSize = maxNonStarters + 2
maxNFCExpansion = 3 // NFC(0x1D160)
maxNFKCExpansion = 18 // NFKC(0xFDFA)
maxByteBufferSize = utf8.UTFMax * maxBufferSize // 128
)
// ssState is used for reporting the segment state after inserting a rune.
// It is returned by streamSafe.next.
type ssState int
const (
// Indicates a rune was successfully added to the segment.
ssSuccess ssState = iota
// Indicates a rune starts a new segment and should not be added.
ssStarter
// Indicates a rune caused a segment overflow and a CGJ should be inserted.
ssOverflow
)
// streamSafe implements the policy of when a CGJ should be inserted.
type streamSafe uint8
// mkStreamSafe is a shorthand for declaring a streamSafe var and calling
// first on it.
func mkStreamSafe(p Properties) streamSafe {
return streamSafe(p.nTrailingNonStarters())
}
// first inserts the first rune of a segment.
func (ss *streamSafe) first(p Properties) {
if *ss != 0 {
panic("!= 0")
}
*ss = streamSafe(p.nTrailingNonStarters())
}
// insert returns a ssState value to indicate whether a rune represented by p
// can be inserted.
func (ss *streamSafe) next(p Properties) ssState {
if *ss > maxNonStarters {
panic("streamSafe was not reset")
}
n := p.nLeadingNonStarters()
if *ss += streamSafe(n); *ss > maxNonStarters {
*ss = 0
return ssOverflow
}
// The Stream-Safe Text Processing prescribes that the counting can stop
// as soon as a starter is encountered. However, there are some starters,
// like Jamo V and T, that can combine with other runes, leaving their
// successive non-starters appended to the previous, possibly causing an
// overflow. We will therefore consider any rune with a non-zero nLead to
// be a non-starter. Note that it always hold that if nLead > 0 then
// nLead == nTrail.
if n == 0 {
*ss = 0
return ssStarter
}
return ssSuccess
}
// backwards is used for checking for overflow and segment starts
// when traversing a string backwards. Users do not need to call first
// for the first rune. The state of the streamSafe retains the count of
// the non-starters loaded.
func (ss *streamSafe) backwards(p Properties) ssState {
if *ss > maxNonStarters {
panic("streamSafe was not reset")
}
c := *ss + streamSafe(p.nTrailingNonStarters())
if c > maxNonStarters {
return ssOverflow
}
*ss = c
if p.nLeadingNonStarters() == 0 {
return ssStarter
}
return ssSuccess
}
func (ss streamSafe) isMax() bool {
return ss == maxNonStarters
}
// GraphemeJoiner is inserted after maxNonStarters non-starter runes.
const GraphemeJoiner = "\u034F"
// reorderBuffer is used to normalize a single segment. Characters inserted with
// insert are decomposed and reordered based on CCC. The compose method can
// be used to recombine characters. Note that the byte buffer does not hold
// the UTF-8 characters in order. Only the rune array is maintained in sorted
// order. flush writes the resulting segment to a byte array.
type reorderBuffer struct {
rune [maxBufferSize]Properties // Per character info.
byte [maxByteBufferSize]byte // UTF-8 buffer. Referenced by runeInfo.pos.
nbyte uint8 // Number or bytes.
ss streamSafe // For limiting length of non-starter sequence.
nrune int // Number of runeInfos.
f formInfo
src input
nsrc int
tmpBytes input
out []byte
flushF func(*reorderBuffer) bool
}
func (rb *reorderBuffer) init(f Form, src []byte) {
rb.f = *formTable[f]
rb.src.setBytes(src)
rb.nsrc = len(src)
rb.ss = 0
}
func (rb *reorderBuffer) initString(f Form, src string) {
rb.f = *formTable[f]
rb.src.setString(src)
rb.nsrc = len(src)
rb.ss = 0
}
func (rb *reorderBuffer) setFlusher(out []byte, f func(*reorderBuffer) bool) {
rb.out = out
rb.flushF = f
}
// reset discards all characters from the buffer.
func (rb *reorderBuffer) reset() {
rb.nrune = 0
rb.nbyte = 0
rb.ss = 0
}
func (rb *reorderBuffer) doFlush() bool {
if rb.f.composing {
rb.compose()
}
res := rb.flushF(rb)
rb.reset()
return res
}
// appendFlush appends the normalized segment to rb.out.
func appendFlush(rb *reorderBuffer) bool {
for i := 0; i < rb.nrune; i++ {
start := rb.rune[i].pos
end := start + rb.rune[i].size
rb.out = append(rb.out, rb.byte[start:end]...)
}
return true
}
// flush appends the normalized segment to out and resets rb.
func (rb *reorderBuffer) flush(out []byte) []byte {
for i := 0; i < rb.nrune; i++ {
start := rb.rune[i].pos
end := start + rb.rune[i].size
out = append(out, rb.byte[start:end]...)
}
rb.reset()
return out
}
// flushCopy copies the normalized segment to buf and resets rb.
// It returns the number of bytes written to buf.
func (rb *reorderBuffer) flushCopy(buf []byte) int {
p := 0
for i := 0; i < rb.nrune; i++ {
runep := rb.rune[i]
p += copy(buf[p:], rb.byte[runep.pos:runep.pos+runep.size])
}
rb.reset()
return p
}
// insertOrdered inserts a rune in the buffer, ordered by Canonical Combining Class.
// It returns false if the buffer is not large enough to hold the rune.
// It is used internally by insert and insertString only.
func (rb *reorderBuffer) insertOrdered(info Properties) {
n := rb.nrune
b := rb.rune[:]
cc := info.ccc
if cc > 0 {
// Find insertion position + move elements to make room.
for ; n > 0; n-- {
if b[n-1].ccc <= cc {
break
}
b[n] = b[n-1]
}
}
rb.nrune += 1
pos := uint8(rb.nbyte)
rb.nbyte += utf8.UTFMax
info.pos = pos
b[n] = info
}
// insertErr is an error code returned by insert. Using this type instead
// of error improves performance up to 20% for many of the benchmarks.
type insertErr int
const (
iSuccess insertErr = -iota
iShortDst
iShortSrc
)
// insertFlush inserts the given rune in the buffer ordered by CCC.
// If a decomposition with multiple segments are encountered, they leading
// ones are flushed.
// It returns a non-zero error code if the rune was not inserted.
func (rb *reorderBuffer) insertFlush(src input, i int, info Properties) insertErr {
if rune := src.hangul(i); rune != 0 {
rb.decomposeHangul(rune)
return iSuccess
}
if info.hasDecomposition() {
return rb.insertDecomposed(info.Decomposition())
}
rb.insertSingle(src, i, info)
return iSuccess
}
// insertUnsafe inserts the given rune in the buffer ordered by CCC.
// It is assumed there is sufficient space to hold the runes. It is the
// responsibility of the caller to ensure this. This can be done by checking
// the state returned by the streamSafe type.
func (rb *reorderBuffer) insertUnsafe(src input, i int, info Properties) {
if rune := src.hangul(i); rune != 0 {
rb.decomposeHangul(rune)
}
if info.hasDecomposition() {
// TODO: inline.
rb.insertDecomposed(info.Decomposition())
} else {
rb.insertSingle(src, i, info)
}
}
// insertDecomposed inserts an entry in to the reorderBuffer for each rune
// in dcomp. dcomp must be a sequence of decomposed UTF-8-encoded runes.
// It flushes the buffer on each new segment start.
func (rb *reorderBuffer) insertDecomposed(dcomp []byte) insertErr {
rb.tmpBytes.setBytes(dcomp)
for i := 0; i < len(dcomp); {
info := rb.f.info(rb.tmpBytes, i)
if info.BoundaryBefore() && rb.nrune > 0 && !rb.doFlush() {
return iShortDst
}
i += copy(rb.byte[rb.nbyte:], dcomp[i:i+int(info.size)])
rb.insertOrdered(info)
}
return iSuccess
}
// insertSingle inserts an entry in the reorderBuffer for the rune at
// position i. info is the runeInfo for the rune at position i.
func (rb *reorderBuffer) insertSingle(src input, i int, info Properties) {
src.copySlice(rb.byte[rb.nbyte:], i, i+int(info.size))
rb.insertOrdered(info)
}
// insertCGJ inserts a Combining Grapheme Joiner (0x034f) into rb.
func (rb *reorderBuffer) insertCGJ() {
rb.insertSingle(input{str: GraphemeJoiner}, 0, Properties{size: uint8(len(GraphemeJoiner))})
}
// appendRune inserts a rune at the end of the buffer. It is used for Hangul.
func (rb *reorderBuffer) appendRune(r rune) {
bn := rb.nbyte
sz := utf8.EncodeRune(rb.byte[bn:], rune(r))
rb.nbyte += utf8.UTFMax
rb.rune[rb.nrune] = Properties{pos: bn, size: uint8(sz)}
rb.nrune++
}
// assignRune sets a rune at position pos. It is used for Hangul and recomposition.
func (rb *reorderBuffer) assignRune(pos int, r rune) {
bn := rb.rune[pos].pos
sz := utf8.EncodeRune(rb.byte[bn:], rune(r))
rb.rune[pos] = Properties{pos: bn, size: uint8(sz)}
}
// runeAt returns the rune at position n. It is used for Hangul and recomposition.
func (rb *reorderBuffer) runeAt(n int) rune {
inf := rb.rune[n]
r, _ := utf8.DecodeRune(rb.byte[inf.pos : inf.pos+inf.size])
return r
}
// bytesAt returns the UTF-8 encoding of the rune at position n.
// It is used for Hangul and recomposition.
func (rb *reorderBuffer) bytesAt(n int) []byte {
inf := rb.rune[n]
return rb.byte[inf.pos : int(inf.pos)+int(inf.size)]
}
// For Hangul we combine algorithmically, instead of using tables.
const (
hangulBase = 0xAC00 // UTF-8(hangulBase) -> EA B0 80
hangulBase0 = 0xEA
hangulBase1 = 0xB0
hangulBase2 = 0x80
hangulEnd = hangulBase + jamoLVTCount // UTF-8(0xD7A4) -> ED 9E A4
hangulEnd0 = 0xED
hangulEnd1 = 0x9E
hangulEnd2 = 0xA4
jamoLBase = 0x1100 // UTF-8(jamoLBase) -> E1 84 00
jamoLBase0 = 0xE1
jamoLBase1 = 0x84
jamoLEnd = 0x1113
jamoVBase = 0x1161
jamoVEnd = 0x1176
jamoTBase = 0x11A7
jamoTEnd = 0x11C3
jamoTCount = 28
jamoVCount = 21
jamoVTCount = 21 * 28
jamoLVTCount = 19 * 21 * 28
)
const hangulUTF8Size = 3
func isHangul(b []byte) bool {
if len(b) < hangulUTF8Size {
return false
}
b0 := b[0]
if b0 < hangulBase0 {
return false
}
b1 := b[1]
switch {
case b0 == hangulBase0:
return b1 >= hangulBase1
case b0 < hangulEnd0:
return true
case b0 > hangulEnd0:
return false
case b1 < hangulEnd1:
return true
}
return b1 == hangulEnd1 && b[2] < hangulEnd2
}
func isHangulString(b string) bool {
if len(b) < hangulUTF8Size {
return false
}
b0 := b[0]
if b0 < hangulBase0 {
return false
}
b1 := b[1]
switch {
case b0 == hangulBase0:
return b1 >= hangulBase1
case b0 < hangulEnd0:
return true
case b0 > hangulEnd0:
return false
case b1 < hangulEnd1:
return true
}
return b1 == hangulEnd1 && b[2] < hangulEnd2
}
// Caller must ensure len(b) >= 2.
func isJamoVT(b []byte) bool {
// True if (rune & 0xff00) == jamoLBase
return b[0] == jamoLBase0 && (b[1]&0xFC) == jamoLBase1
}
func isHangulWithoutJamoT(b []byte) bool {
c, _ := utf8.DecodeRune(b)
c -= hangulBase
return c < jamoLVTCount && c%jamoTCount == 0
}
// decomposeHangul writes the decomposed Hangul to buf and returns the number
// of bytes written. len(buf) should be at least 9.
func decomposeHangul(buf []byte, r rune) int {
const JamoUTF8Len = 3
r -= hangulBase
x := r % jamoTCount
r /= jamoTCount
utf8.EncodeRune(buf, jamoLBase+r/jamoVCount)
utf8.EncodeRune(buf[JamoUTF8Len:], jamoVBase+r%jamoVCount)
if x != 0 {
utf8.EncodeRune(buf[2*JamoUTF8Len:], jamoTBase+x)
return 3 * JamoUTF8Len
}
return 2 * JamoUTF8Len
}
// decomposeHangul algorithmically decomposes a Hangul rune into
// its Jamo components.
// See http://unicode.org/reports/tr15/#Hangul for details on decomposing Hangul.
func (rb *reorderBuffer) decomposeHangul(r rune) {
r -= hangulBase
x := r % jamoTCount
r /= jamoTCount
rb.appendRune(jamoLBase + r/jamoVCount)
rb.appendRune(jamoVBase + r%jamoVCount)
if x != 0 {
rb.appendRune(jamoTBase + x)
}
}
// combineHangul algorithmically combines Jamo character components into Hangul.
// See http://unicode.org/reports/tr15/#Hangul for details on combining Hangul.
func (rb *reorderBuffer) combineHangul(s, i, k int) {
b := rb.rune[:]
bn := rb.nrune
for ; i < bn; i++ {
cccB := b[k-1].ccc
cccC := b[i].ccc
if cccB == 0 {
s = k - 1
}
if s != k-1 && cccB >= cccC {
// b[i] is blocked by greater-equal cccX below it
b[k] = b[i]
k++
} else {
l := rb.runeAt(s) // also used to compare to hangulBase
v := rb.runeAt(i) // also used to compare to jamoT
switch {
case jamoLBase <= l && l < jamoLEnd &&
jamoVBase <= v && v < jamoVEnd:
// 11xx plus 116x to LV
rb.assignRune(s, hangulBase+
(l-jamoLBase)*jamoVTCount+(v-jamoVBase)*jamoTCount)
case hangulBase <= l && l < hangulEnd &&
jamoTBase < v && v < jamoTEnd &&
((l-hangulBase)%jamoTCount) == 0:
// ACxx plus 11Ax to LVT
rb.assignRune(s, l+v-jamoTBase)
default:
b[k] = b[i]
k++
}
}
}
rb.nrune = k
}
// compose recombines the runes in the buffer.
// It should only be used to recompose a single segment, as it will not
// handle alternations between Hangul and non-Hangul characters correctly.
func (rb *reorderBuffer) compose() {
// UAX #15, section X5 , including Corrigendum #5
// "In any character sequence beginning with starter S, a character C is
// blocked from S if and only if there is some character B between S
// and C, and either B is a starter or it has the same or higher
// combining class as C."
bn := rb.nrune
if bn == 0 {
return
}
k := 1
b := rb.rune[:]
for s, i := 0, 1; i < bn; i++ {
if isJamoVT(rb.bytesAt(i)) {
// Redo from start in Hangul mode. Necessary to support
// U+320E..U+321E in NFKC mode.
rb.combineHangul(s, i, k)
return
}
ii := b[i]
// We can only use combineForward as a filter if we later
// get the info for the combined character. This is more
// expensive than using the filter. Using combinesBackward()
// is safe.
if ii.combinesBackward() {
cccB := b[k-1].ccc
cccC := ii.ccc
blocked := false // b[i] blocked by starter or greater or equal CCC?
if cccB == 0 {
s = k - 1
} else {
blocked = s != k-1 && cccB >= cccC
}
if !blocked {
combined := combine(rb.runeAt(s), rb.runeAt(i))
if combined != 0 {
rb.assignRune(s, combined)
continue
}
}
}
b[k] = b[i]
k++
}
rb.nrune = k
}
@@ -1,130 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import "testing"
// TestCase is used for most tests.
type TestCase struct {
in []rune
out []rune
}
func runTests(t *testing.T, name string, fm Form, tests []TestCase) {
rb := reorderBuffer{}
rb.init(fm, nil)
for i, test := range tests {
rb.setFlusher(nil, appendFlush)
for j, rune := range test.in {
b := []byte(string(rune))
src := inputBytes(b)
info := rb.f.info(src, 0)
if j == 0 {
rb.ss.first(info)
} else {
rb.ss.next(info)
}
if rb.insertFlush(src, 0, info) < 0 {
t.Errorf("%s:%d: insert failed for rune %d", name, i, j)
}
}
rb.doFlush()
was := string(rb.out)
want := string(test.out)
if len(was) != len(want) {
t.Errorf("%s:%d: length = %d; want %d", name, i, len(was), len(want))
}
if was != want {
k, pfx := pidx(was, want)
t.Errorf("%s:%d: \nwas %s%+q; \nwant %s%+q", name, i, pfx, was[k:], pfx, want[k:])
}
}
}
func TestFlush(t *testing.T) {
const (
hello = "Hello "
world = "world!"
)
buf := make([]byte, maxByteBufferSize)
p := copy(buf, hello)
out := buf[p:]
rb := reorderBuffer{}
rb.initString(NFC, world)
if i := rb.flushCopy(out); i != 0 {
t.Errorf("wrote bytes on flush of empty buffer. (len(out) = %d)", i)
}
for i := range world {
// No need to set streamSafe values for this test.
rb.insertFlush(rb.src, i, rb.f.info(rb.src, i))
n := rb.flushCopy(out)
out = out[n:]
p += n
}
was := buf[:p]
want := hello + world
if string(was) != want {
t.Errorf(`output after flush was "%s"; want "%s"`, string(was), want)
}
if rb.nrune != 0 {
t.Errorf("non-null size of info buffer (rb.nrune == %d)", rb.nrune)
}
if rb.nbyte != 0 {
t.Errorf("non-null size of byte buffer (rb.nbyte == %d)", rb.nbyte)
}
}
var insertTests = []TestCase{
{[]rune{'a'}, []rune{'a'}},
{[]rune{0x300}, []rune{0x300}},
{[]rune{0x300, 0x316}, []rune{0x316, 0x300}}, // CCC(0x300)==230; CCC(0x316)==220
{[]rune{0x316, 0x300}, []rune{0x316, 0x300}},
{[]rune{0x41, 0x316, 0x300}, []rune{0x41, 0x316, 0x300}},
{[]rune{0x41, 0x300, 0x316}, []rune{0x41, 0x316, 0x300}},
{[]rune{0x300, 0x316, 0x41}, []rune{0x316, 0x300, 0x41}},
{[]rune{0x41, 0x300, 0x40, 0x316}, []rune{0x41, 0x300, 0x40, 0x316}},
}
func TestInsert(t *testing.T) {
runTests(t, "TestInsert", NFD, insertTests)
}
var decompositionNFDTest = []TestCase{
{[]rune{0xC0}, []rune{0x41, 0x300}},
{[]rune{0xAC00}, []rune{0x1100, 0x1161}},
{[]rune{0x01C4}, []rune{0x01C4}},
{[]rune{0x320E}, []rune{0x320E}},
{[]rune("음ẻ과"), []rune{0x110B, 0x1173, 0x11B7, 0x65, 0x309, 0x1100, 0x116A}},
}
var decompositionNFKDTest = []TestCase{
{[]rune{0xC0}, []rune{0x41, 0x300}},
{[]rune{0xAC00}, []rune{0x1100, 0x1161}},
{[]rune{0x01C4}, []rune{0x44, 0x5A, 0x030C}},
{[]rune{0x320E}, []rune{0x28, 0x1100, 0x1161, 0x29}},
}
func TestDecomposition(t *testing.T) {
runTests(t, "TestDecompositionNFD", NFD, decompositionNFDTest)
runTests(t, "TestDecompositionNFKD", NFKD, decompositionNFKDTest)
}
var compositionTest = []TestCase{
{[]rune{0x41, 0x300}, []rune{0xC0}},
{[]rune{0x41, 0x316}, []rune{0x41, 0x316}},
{[]rune{0x41, 0x300, 0x35D}, []rune{0xC0, 0x35D}},
{[]rune{0x41, 0x316, 0x300}, []rune{0xC0, 0x316}},
// blocking starter
{[]rune{0x41, 0x316, 0x40, 0x300}, []rune{0x41, 0x316, 0x40, 0x300}},
{[]rune{0x1100, 0x1161}, []rune{0xAC00}},
// parenthesized Hangul, alternate between ASCII and Hangul.
{[]rune{0x28, 0x1100, 0x1161, 0x29}, []rune{0x28, 0xAC00, 0x29}},
}
func TestComposition(t *testing.T) {
runTests(t, "TestComposition", NFC, compositionTest)
}
@@ -1,82 +0,0 @@
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm_test
import (
"bytes"
"fmt"
"unicode/utf8"
"code.google.com/p/go.text/unicode/norm"
)
// EqualSimple uses a norm.Iter to compare two non-normalized
// strings for equivalence.
func EqualSimple(a, b string) bool {
var ia, ib norm.Iter
ia.InitString(norm.NFKD, a)
ib.InitString(norm.NFKD, b)
for !ia.Done() && !ib.Done() {
if !bytes.Equal(ia.Next(), ib.Next()) {
return false
}
}
return ia.Done() && ib.Done()
}
// FindPrefix finds the longest common prefix of ASCII characters
// of a and b.
func FindPrefix(a, b string) int {
i := 0
for ; i < len(a) && i < len(b) && a[i] < utf8.RuneSelf && a[i] == b[i]; i++ {
}
return i
}
// EqualOpt is like EqualSimple, but optimizes the special
// case for ASCII characters.
func EqualOpt(a, b string) bool {
n := FindPrefix(a, b)
a, b = a[n:], b[n:]
var ia, ib norm.Iter
ia.InitString(norm.NFKD, a)
ib.InitString(norm.NFKD, b)
for !ia.Done() && !ib.Done() {
if !bytes.Equal(ia.Next(), ib.Next()) {
return false
}
if n := int64(FindPrefix(a[ia.Pos():], b[ib.Pos():])); n != 0 {
ia.Seek(n, 1)
ib.Seek(n, 1)
}
}
return ia.Done() && ib.Done()
}
var compareTests = []struct{ a, b string }{
{"aaa", "aaa"},
{"aaa", "aab"},
{"a\u0300a", "\u00E0a"},
{"a\u0300\u0320b", "a\u0320\u0300b"},
{"\u1E0A\u0323", "\x44\u0323\u0307"},
// A character that decomposes into multiple segments
// spans several iterations.
{"\u3304", "\u30A4\u30CB\u30F3\u30AF\u3099"},
}
func ExampleIter() {
for i, t := range compareTests {
r0 := EqualSimple(t.a, t.b)
r1 := EqualOpt(t.a, t.b)
fmt.Printf("%d: %v %v\n", i, r0, r1)
}
// Output:
// 0: true true
// 1: false false
// 2: true true
// 3: true true
// 4: true true
// 5: true true
}
@@ -1,256 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
// This file contains Form-specific logic and wrappers for data in tables.go.
// Rune info is stored in a separate trie per composing form. A composing form
// and its corresponding decomposing form share the same trie. Each trie maps
// a rune to a uint16. The values take two forms. For v >= 0x8000:
// bits
// 15: 1 (inverse of NFD_QD bit of qcInfo)
// 13..7: qcInfo (see below). isYesD is always true (no decompostion).
// 6..0: ccc (compressed CCC value).
// For v < 0x8000, the respective rune has a decomposition and v is an index
// into a byte array of UTF-8 decomposition sequences and additional info and
// has the form:
// <header> <decomp_byte>* [<tccc> [<lccc>]]
// The header contains the number of bytes in the decomposition (excluding this
// length byte). The two most significant bits of this length byte correspond
// to bit 5 and 4 of qcInfo (see below). The byte sequence itself starts at v+1.
// The byte sequence is followed by a trailing and leading CCC if the values
// for these are not zero. The value of v determines which ccc are appended
// to the sequences. For v < firstCCC, there are none, for v >= firstCCC,
// the sequence is followed by a trailing ccc, and for v >= firstLeadingCC
// there is an additional leading ccc. The value of tccc itself is the
// trailing CCC shifted left 2 bits. The two least-significant bits of tccc
// are the number of trailing non-starters.
const (
qcInfoMask = 0x3F // to clear all but the relevant bits in a qcInfo
headerLenMask = 0x3F // extract the length value from the header byte
headerFlagsMask = 0xC0 // extract the qcInfo bits from the header byte
)
// Properties provides access to normalization properties of a rune.
type Properties struct {
pos uint8 // start position in reorderBuffer; used in composition.go
size uint8 // length of UTF-8 encoding of this rune
ccc uint8 // leading canonical combining class (ccc if not decomposition)
tccc uint8 // trailing canonical combining class (ccc if not decomposition)
nLead uint8 // number of leading non-starters.
flags qcInfo // quick check flags
index uint16
}
// functions dispatchable per form
type lookupFunc func(b input, i int) Properties
// formInfo holds Form-specific functions and tables.
type formInfo struct {
form Form
composing, compatibility bool // form type
info lookupFunc
nextMain iterFunc
}
var formTable []*formInfo
func init() {
formTable = make([]*formInfo, 4)
for i := range formTable {
f := &formInfo{}
formTable[i] = f
f.form = Form(i)
if Form(i) == NFKD || Form(i) == NFKC {
f.compatibility = true
f.info = lookupInfoNFKC
} else {
f.info = lookupInfoNFC
}
f.nextMain = nextDecomposed
if Form(i) == NFC || Form(i) == NFKC {
f.nextMain = nextComposed
f.composing = true
}
}
}
// We do not distinguish between boundaries for NFC, NFD, etc. to avoid
// unexpected behavior for the user. For example, in NFD, there is a boundary
// after 'a'. However, 'a' might combine with modifiers, so from the application's
// perspective it is not a good boundary. We will therefore always use the
// boundaries for the combining variants.
// BoundaryBefore returns true if this rune starts a new segment and
// cannot combine with any rune on the left.
func (p Properties) BoundaryBefore() bool {
if p.ccc == 0 && !p.combinesBackward() {
return true
}
// We assume that the CCC of the first character in a decomposition
// is always non-zero if different from info.ccc and that we can return
// false at this point. This is verified by maketables.
return false
}
// BoundaryAfter returns true if runes cannot combine with or otherwise
// interact with this or previous runes.
func (p Properties) BoundaryAfter() bool {
// TODO: loosen these conditions.
return p.isInert()
}
// We pack quick check data in 4 bits:
// 5: Combines forward (0 == false, 1 == true)
// 4..3: NFC_QC Yes(00), No (10), or Maybe (11)
// 2: NFD_QC Yes (0) or No (1). No also means there is a decomposition.
// 1..0: Number of trailing non-starters.
//
// When all 4 bits are zero, the character is inert, meaning it is never
// influenced by normalization.
type qcInfo uint8
func (p Properties) isYesC() bool { return p.flags&0x10 == 0 }
func (p Properties) isYesD() bool { return p.flags&0x4 == 0 }
func (p Properties) combinesForward() bool { return p.flags&0x20 != 0 }
func (p Properties) combinesBackward() bool { return p.flags&0x8 != 0 } // == isMaybe
func (p Properties) hasDecomposition() bool { return p.flags&0x4 != 0 } // == isNoD
func (p Properties) isInert() bool {
return p.flags&qcInfoMask == 0 && p.ccc == 0
}
func (p Properties) multiSegment() bool {
return p.index >= firstMulti && p.index < endMulti
}
func (p Properties) nLeadingNonStarters() uint8 {
return p.nLead
}
func (p Properties) nTrailingNonStarters() uint8 {
return uint8(p.flags & 0x03)
}
// Decomposition returns the decomposition for the underlying rune
// or nil if there is none.
func (p Properties) Decomposition() []byte {
// TODO: create the decomposition for Hangul?
if p.index == 0 {
return nil
}
i := p.index
n := decomps[i] & headerLenMask
i++
return decomps[i : i+uint16(n)]
}
// Size returns the length of UTF-8 encoding of the rune.
func (p Properties) Size() int {
return int(p.size)
}
// CCC returns the canonical combining class of the underlying rune.
func (p Properties) CCC() uint8 {
if p.index >= firstCCCZeroExcept {
return 0
}
return ccc[p.ccc]
}
// LeadCCC returns the CCC of the first rune in the decomposition.
// If there is no decomposition, LeadCCC equals CCC.
func (p Properties) LeadCCC() uint8 {
return ccc[p.ccc]
}
// TrailCCC returns the CCC of the last rune in the decomposition.
// If there is no decomposition, TrailCCC equals CCC.
func (p Properties) TrailCCC() uint8 {
return ccc[p.tccc]
}
// Recomposition
// We use 32-bit keys instead of 64-bit for the two codepoint keys.
// This clips off the bits of three entries, but we know this will not
// result in a collision. In the unlikely event that changes to
// UnicodeData.txt introduce collisions, the compiler will catch it.
// Note that the recomposition map for NFC and NFKC are identical.
// combine returns the combined rune or 0 if it doesn't exist.
func combine(a, b rune) rune {
key := uint32(uint16(a))<<16 + uint32(uint16(b))
return recompMap[key]
}
func lookupInfoNFC(b input, i int) Properties {
v, sz := b.charinfoNFC(i)
return compInfo(v, sz)
}
func lookupInfoNFKC(b input, i int) Properties {
v, sz := b.charinfoNFKC(i)
return compInfo(v, sz)
}
// Properties returns properties for the first rune in s.
func (f Form) Properties(s []byte) Properties {
if f == NFC || f == NFD {
return compInfo(nfcTrie.lookup(s))
}
return compInfo(nfkcTrie.lookup(s))
}
// PropertiesString returns properties for the first rune in s.
func (f Form) PropertiesString(s string) Properties {
if f == NFC || f == NFD {
return compInfo(nfcTrie.lookupString(s))
}
return compInfo(nfkcTrie.lookupString(s))
}
// compInfo converts the information contained in v and sz
// to a Properties. See the comment at the top of the file
// for more information on the format.
func compInfo(v uint16, sz int) Properties {
if v == 0 {
return Properties{size: uint8(sz)}
} else if v >= 0x8000 {
p := Properties{
size: uint8(sz),
ccc: uint8(v),
tccc: uint8(v),
flags: qcInfo(v >> 8),
}
if p.ccc > 0 || p.combinesBackward() {
p.nLead = uint8(p.flags & 0x3)
}
return p
}
// has decomposition
h := decomps[v]
f := (qcInfo(h&headerFlagsMask) >> 2) | 0x4
p := Properties{size: uint8(sz), flags: f, index: v}
if v >= firstCCC {
v += uint16(h&headerLenMask) + 1
c := decomps[v]
p.tccc = c >> 2
p.flags |= qcInfo(c & 0x3)
if v >= firstLeadingCCC {
p.nLead = c & 0x3
if v >= firstStarterWithNLead {
// We were tricked. Remove the decomposition.
p.flags &= 0x03
p.index = 0
return p
}
p.ccc = decomps[v+1]
}
}
return p
}
@@ -1,54 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build test
package norm
import "testing"
func TestProperties(t *testing.T) {
var d runeData
CK := [2]string{"C", "K"}
for k, r := 1, rune(0); r < 0x2ffff; r++ {
if k < len(testData) && r == testData[k].r {
d = testData[k]
k++
}
s := string(r)
for j, p := range []Properties{NFC.PropertiesString(s), NFKC.PropertiesString(s)} {
f := d.f[j]
if p.CCC() != d.ccc {
t.Errorf("%U: ccc(%s): was %d; want %d %X", r, CK[j], p.CCC(), d.ccc, p.index)
}
if p.isYesC() != (f.qc == Yes) {
t.Errorf("%U: YesC(%s): was %v; want %v", r, CK[j], p.isYesC(), f.qc == Yes)
}
if p.combinesBackward() != (f.qc == Maybe) {
t.Errorf("%U: combines backwards(%s): was %v; want %v", r, CK[j], p.combinesBackward(), f.qc == Maybe)
}
if p.nLeadingNonStarters() != d.nLead {
t.Errorf("%U: nLead(%s): was %d; want %d %#v %#v", r, CK[j], p.nLeadingNonStarters(), d.nLead, p, d)
}
if p.nTrailingNonStarters() != d.nTrail {
t.Errorf("%U: nTrail(%s): was %d; want %d %#v %#v", r, CK[j], p.nTrailingNonStarters(), d.nTrail, p, d)
}
if p.combinesForward() != f.combinesForward {
t.Errorf("%U: combines forward(%s): was %v; want %v %#v", r, CK[j], p.combinesForward(), f.combinesForward, p)
}
// Skip Hangul as it is algorithmically computed.
if r >= hangulBase && r < hangulEnd {
continue
}
if p.hasDecomposition() {
if has := f.decomposition != ""; !has {
t.Errorf("%U: hasDecomposition(%s): was %v; want %v", r, CK[j], p.hasDecomposition(), has)
}
if string(p.Decomposition()) != f.decomposition {
t.Errorf("%U: decomp(%s): was %+q; want %+q", r, CK[j], p.Decomposition(), f.decomposition)
}
}
}
}
}
-105
View File
@@ -1,105 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import "unicode/utf8"
type input struct {
str string
bytes []byte
}
func inputBytes(str []byte) input {
return input{bytes: str}
}
func inputString(str string) input {
return input{str: str}
}
func (in *input) setBytes(str []byte) {
in.str = ""
in.bytes = str
}
func (in *input) setString(str string) {
in.str = str
in.bytes = nil
}
func (in *input) _byte(p int) byte {
if in.bytes == nil {
return in.str[p]
}
return in.bytes[p]
}
func (in *input) skipASCII(p, max int) int {
if in.bytes == nil {
for ; p < max && in.str[p] < utf8.RuneSelf; p++ {
}
} else {
for ; p < max && in.bytes[p] < utf8.RuneSelf; p++ {
}
}
return p
}
func (in *input) skipContinuationBytes(p int) int {
if in.bytes == nil {
for ; p < len(in.str) && !utf8.RuneStart(in.str[p]); p++ {
}
} else {
for ; p < len(in.bytes) && !utf8.RuneStart(in.bytes[p]); p++ {
}
}
return p
}
func (in *input) appendSlice(buf []byte, b, e int) []byte {
if in.bytes != nil {
return append(buf, in.bytes[b:e]...)
}
for i := b; i < e; i++ {
buf = append(buf, in.str[i])
}
return buf
}
func (in *input) copySlice(buf []byte, b, e int) int {
if in.bytes == nil {
return copy(buf, in.str[b:e])
}
return copy(buf, in.bytes[b:e])
}
func (in *input) charinfoNFC(p int) (uint16, int) {
if in.bytes == nil {
return nfcTrie.lookupString(in.str[p:])
}
return nfcTrie.lookup(in.bytes[p:])
}
func (in *input) charinfoNFKC(p int) (uint16, int) {
if in.bytes == nil {
return nfkcTrie.lookupString(in.str[p:])
}
return nfkcTrie.lookup(in.bytes[p:])
}
func (in *input) hangul(p int) (r rune) {
if in.bytes == nil {
if !isHangulString(in.str[p:]) {
return 0
}
r, _ = utf8.DecodeRuneInString(in.str[p:])
} else {
if !isHangul(in.bytes[p:]) {
return 0
}
r, _ = utf8.DecodeRune(in.bytes[p:])
}
return r
}
-448
View File
@@ -1,448 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import (
"fmt"
"unicode/utf8"
)
const MaxSegmentSize = maxByteBufferSize
// An Iter iterates over a string or byte slice, while normalizing it
// to a given Form.
type Iter struct {
rb reorderBuffer
buf [maxByteBufferSize]byte
info Properties // first character saved from previous iteration
next iterFunc // implementation of next depends on form
asciiF iterFunc
p int // current position in input source
multiSeg []byte // remainder of multi-segment decomposition
}
type iterFunc func(*Iter) []byte
// Init initializes i to iterate over src after normalizing it to Form f.
func (i *Iter) Init(f Form, src []byte) {
i.p = 0
if len(src) == 0 {
i.setDone()
i.rb.nsrc = 0
return
}
i.multiSeg = nil
i.rb.init(f, src)
i.next = i.rb.f.nextMain
i.asciiF = nextASCIIBytes
i.info = i.rb.f.info(i.rb.src, i.p)
}
// InitString initializes i to iterate over src after normalizing it to Form f.
func (i *Iter) InitString(f Form, src string) {
i.p = 0
if len(src) == 0 {
i.setDone()
i.rb.nsrc = 0
return
}
i.multiSeg = nil
i.rb.initString(f, src)
i.next = i.rb.f.nextMain
i.asciiF = nextASCIIString
i.info = i.rb.f.info(i.rb.src, i.p)
}
// Seek sets the segment to be returned by the next call to Next to start
// at position p. It is the responsibility of the caller to set p to the
// start of a UTF8 rune.
func (i *Iter) Seek(offset int64, whence int) (int64, error) {
var abs int64
switch whence {
case 0:
abs = offset
case 1:
abs = int64(i.p) + offset
case 2:
abs = int64(i.rb.nsrc) + offset
default:
return 0, fmt.Errorf("norm: invalid whence")
}
if abs < 0 {
return 0, fmt.Errorf("norm: negative position")
}
if int(abs) >= i.rb.nsrc {
i.setDone()
return int64(i.p), nil
}
i.p = int(abs)
i.multiSeg = nil
i.next = i.rb.f.nextMain
i.info = i.rb.f.info(i.rb.src, i.p)
return abs, nil
}
// returnSlice returns a slice of the underlying input type as a byte slice.
// If the underlying is of type []byte, it will simply return a slice.
// If the underlying is of type string, it will copy the slice to the buffer
// and return that.
func (i *Iter) returnSlice(a, b int) []byte {
if i.rb.src.bytes == nil {
return i.buf[:copy(i.buf[:], i.rb.src.str[a:b])]
}
return i.rb.src.bytes[a:b]
}
// Pos returns the byte position at which the next call to Next will commence processing.
func (i *Iter) Pos() int {
return i.p
}
func (i *Iter) setDone() {
i.next = nextDone
i.p = i.rb.nsrc
}
// Done returns true if there is no more input to process.
func (i *Iter) Done() bool {
return i.p >= i.rb.nsrc
}
// Next returns f(i.input[i.Pos():n]), where n is a boundary of i.input.
// For any input a and b for which f(a) == f(b), subsequent calls
// to Next will return the same segments.
// Modifying runes are grouped together with the preceding starter, if such a starter exists.
// Although not guaranteed, n will typically be the smallest possible n.
func (i *Iter) Next() []byte {
return i.next(i)
}
func nextASCIIBytes(i *Iter) []byte {
p := i.p + 1
if p >= i.rb.nsrc {
i.setDone()
return i.rb.src.bytes[i.p:p]
}
if i.rb.src.bytes[p] < utf8.RuneSelf {
p0 := i.p
i.p = p
return i.rb.src.bytes[p0:p]
}
i.info = i.rb.f.info(i.rb.src, i.p)
i.next = i.rb.f.nextMain
return i.next(i)
}
func nextASCIIString(i *Iter) []byte {
p := i.p + 1
if p >= i.rb.nsrc {
i.buf[0] = i.rb.src.str[i.p]
i.setDone()
return i.buf[:1]
}
if i.rb.src.str[p] < utf8.RuneSelf {
i.buf[0] = i.rb.src.str[i.p]
i.p = p
return i.buf[:1]
}
i.info = i.rb.f.info(i.rb.src, i.p)
i.next = i.rb.f.nextMain
return i.next(i)
}
func nextHangul(i *Iter) []byte {
p := i.p
next := p + hangulUTF8Size
if next >= i.rb.nsrc {
i.setDone()
} else if i.rb.src.hangul(next) == 0 {
i.info = i.rb.f.info(i.rb.src, i.p)
i.next = i.rb.f.nextMain
return i.next(i)
}
i.p = next
return i.buf[:decomposeHangul(i.buf[:], i.rb.src.hangul(p))]
}
func nextDone(i *Iter) []byte {
return nil
}
// nextMulti is used for iterating over multi-segment decompositions
// for decomposing normal forms.
func nextMulti(i *Iter) []byte {
j := 0
d := i.multiSeg
// skip first rune
for j = 1; j < len(d) && !utf8.RuneStart(d[j]); j++ {
}
for j < len(d) {
info := i.rb.f.info(input{bytes: d}, j)
if info.BoundaryBefore() {
i.multiSeg = d[j:]
return d[:j]
}
j += int(info.size)
}
// treat last segment as normal decomposition
i.next = i.rb.f.nextMain
return i.next(i)
}
// nextMultiNorm is used for iterating over multi-segment decompositions
// for composing normal forms.
func nextMultiNorm(i *Iter) []byte {
j := 0
d := i.multiSeg
for j < len(d) {
info := i.rb.f.info(input{bytes: d}, j)
if info.BoundaryBefore() {
i.rb.compose()
seg := i.buf[:i.rb.flushCopy(i.buf[:])]
i.rb.ss.first(info)
i.rb.insertUnsafe(input{bytes: d}, j, info)
i.multiSeg = d[j+int(info.size):]
return seg
}
i.rb.ss.next(info)
i.rb.insertUnsafe(input{bytes: d}, j, info)
j += int(info.size)
}
i.multiSeg = nil
i.next = nextComposed
return doNormComposed(i)
}
// nextDecomposed is the implementation of Next for forms NFD and NFKD.
func nextDecomposed(i *Iter) (next []byte) {
outp := 0
inCopyStart, outCopyStart := i.p, 0
ss := mkStreamSafe(i.info)
for {
if sz := int(i.info.size); sz <= 1 {
p := i.p
i.p++ // ASCII or illegal byte. Either way, advance by 1.
if i.p >= i.rb.nsrc {
i.setDone()
return i.returnSlice(p, i.p)
} else if i.rb.src._byte(i.p) < utf8.RuneSelf {
i.next = i.asciiF
return i.returnSlice(p, i.p)
}
outp++
} else if d := i.info.Decomposition(); d != nil {
// Note: If leading CCC != 0, then len(d) == 2 and last is also non-zero.
// Case 1: there is a leftover to copy. In this case the decomposition
// must begin with a modifier and should always be appended.
// Case 2: no leftover. Simply return d if followed by a ccc == 0 value.
p := outp + len(d)
if outp > 0 {
i.rb.src.copySlice(i.buf[outCopyStart:], inCopyStart, i.p)
if p > len(i.buf) {
return i.buf[:outp]
}
} else if i.info.multiSegment() {
// outp must be 0 as multi-segment decompositions always
// start a new segment.
if i.multiSeg == nil {
i.multiSeg = d
i.next = nextMulti
return nextMulti(i)
}
// We are in the last segment. Treat as normal decomposition.
d = i.multiSeg
i.multiSeg = nil
p = len(d)
}
prevCC := i.info.tccc
if i.p += sz; i.p >= i.rb.nsrc {
i.setDone()
i.info = Properties{} // Force BoundaryBefore to succeed.
} else {
i.info = i.rb.f.info(i.rb.src, i.p)
}
switch ss.next(i.info) {
case ssOverflow:
i.next = nextCGJDecompose
fallthrough
case ssStarter:
if outp > 0 {
copy(i.buf[outp:], d)
return i.buf[:p]
}
return d
}
copy(i.buf[outp:], d)
outp = p
inCopyStart, outCopyStart = i.p, outp
if i.info.ccc < prevCC {
goto doNorm
}
continue
} else if r := i.rb.src.hangul(i.p); r != 0 {
outp = decomposeHangul(i.buf[:], r)
i.p += hangulUTF8Size
inCopyStart, outCopyStart = i.p, outp
if i.p >= i.rb.nsrc {
i.setDone()
break
} else if i.rb.src.hangul(i.p) != 0 {
i.next = nextHangul
return i.buf[:outp]
}
} else {
p := outp + sz
if p > len(i.buf) {
break
}
outp = p
i.p += sz
}
if i.p >= i.rb.nsrc {
i.setDone()
break
}
prevCC := i.info.tccc
i.info = i.rb.f.info(i.rb.src, i.p)
if v := ss.next(i.info); v == ssStarter {
break
} else if v == ssOverflow {
i.next = nextCGJDecompose
break
}
if i.info.ccc < prevCC {
goto doNorm
}
}
if outCopyStart == 0 {
return i.returnSlice(inCopyStart, i.p)
} else if inCopyStart < i.p {
i.rb.src.copySlice(i.buf[outCopyStart:], inCopyStart, i.p)
}
return i.buf[:outp]
doNorm:
// Insert what we have decomposed so far in the reorderBuffer.
// As we will only reorder, there will always be enough room.
i.rb.src.copySlice(i.buf[outCopyStart:], inCopyStart, i.p)
i.rb.insertDecomposed(i.buf[0:outp])
return doNormDecomposed(i)
}
func doNormDecomposed(i *Iter) []byte {
for {
if s := i.rb.ss.next(i.info); s == ssOverflow {
i.next = nextCGJDecompose
break
}
i.rb.insertUnsafe(i.rb.src, i.p, i.info)
if i.p += int(i.info.size); i.p >= i.rb.nsrc {
i.setDone()
break
}
i.info = i.rb.f.info(i.rb.src, i.p)
if i.info.ccc == 0 {
break
}
}
// new segment or too many combining characters: exit normalization
return i.buf[:i.rb.flushCopy(i.buf[:])]
}
func nextCGJDecompose(i *Iter) []byte {
i.rb.ss = 0
i.rb.insertCGJ()
i.next = nextDecomposed
buf := doNormDecomposed(i)
return buf
}
// nextComposed is the implementation of Next for forms NFC and NFKC.
func nextComposed(i *Iter) []byte {
outp, startp := 0, i.p
var prevCC uint8
ss := mkStreamSafe(i.info)
for {
if !i.info.isYesC() {
goto doNorm
}
prevCC = i.info.tccc
sz := int(i.info.size)
if sz == 0 {
sz = 1 // illegal rune: copy byte-by-byte
}
p := outp + sz
if p > len(i.buf) {
break
}
outp = p
i.p += sz
if i.p >= i.rb.nsrc {
i.setDone()
break
} else if i.rb.src._byte(i.p) < utf8.RuneSelf {
i.next = i.asciiF
break
}
i.info = i.rb.f.info(i.rb.src, i.p)
if v := ss.next(i.info); v == ssStarter {
break
} else if v == ssOverflow {
i.next = nextCGJCompose
break
}
if i.info.ccc < prevCC {
goto doNorm
}
}
return i.returnSlice(startp, i.p)
doNorm:
i.p = startp
i.info = i.rb.f.info(i.rb.src, i.p)
if i.info.multiSegment() {
d := i.info.Decomposition()
info := i.rb.f.info(input{bytes: d}, 0)
i.rb.insertUnsafe(input{bytes: d}, 0, info)
i.multiSeg = d[int(info.size):]
i.next = nextMultiNorm
return nextMultiNorm(i)
}
i.rb.ss.first(i.info)
i.rb.insertUnsafe(i.rb.src, i.p, i.info)
return doNormComposed(i)
}
func doNormComposed(i *Iter) []byte {
// First rune should already be inserted.
for {
if i.p += int(i.info.size); i.p >= i.rb.nsrc {
i.setDone()
break
}
i.info = i.rb.f.info(i.rb.src, i.p)
if s := i.rb.ss.next(i.info); s == ssStarter {
break
} else if s == ssOverflow {
i.next = nextCGJCompose
break
}
i.rb.insertUnsafe(i.rb.src, i.p, i.info)
}
i.rb.compose()
seg := i.buf[:i.rb.flushCopy(i.buf[:])]
return seg
}
func nextCGJCompose(i *Iter) []byte {
i.rb.ss = 0 // instead of first
i.rb.insertCGJ()
i.next = nextComposed
// Note that we treat any rune with nLeadingNonStarters > 0 as a non-starter,
// even if they are not. This is particularly dubious for U+FF9E and UFF9A.
// If we ever change that, insert a check here.
i.rb.ss.first(i.info)
i.rb.insertUnsafe(i.rb.src, i.p, i.info)
return doNormComposed(i)
}
@@ -1,98 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import (
"strings"
"testing"
)
func doIterNorm(f Form, s string) []byte {
acc := []byte{}
i := Iter{}
i.InitString(f, s)
for !i.Done() {
acc = append(acc, i.Next()...)
}
return acc
}
func TestIterNext(t *testing.T) {
runNormTests(t, "IterNext", func(f Form, out []byte, s string) []byte {
return doIterNorm(f, string(append(out, s...)))
})
}
type SegmentTest struct {
in string
out []string
}
var segmentTests = []SegmentTest{
{"\u1E0A\u0323a", []string{"\x44\u0323\u0307", "a", ""}},
{rep('a', segSize), append(strings.Split(rep('a', segSize), ""), "")},
{rep('a', segSize+2), append(strings.Split(rep('a', segSize+2), ""), "")},
{rep('a', segSize) + "\u0300aa",
append(strings.Split(rep('a', segSize-1), ""), "a\u0300", "a", "a", "")},
// U+0f73 is NOT treated as a starter as it is a modifier
{"a" + grave(29) + "\u0f73", []string{"a" + grave(29), cgj + "\u0f73"}},
{"a\u0f73", []string{"a\u0f73"}},
// U+ff9e is treated as a non-starter.
// TODO: should we? Note that this will only affect iteration, as whether
// or not we do so does not affect the normalization output and will either
// way result in consistent iteration output.
{"a" + grave(30) + "\uff9e", []string{"a" + grave(30), cgj + "\uff9e"}},
{"a\uff9e", []string{"a\uff9e"}},
}
var segmentTestsK = []SegmentTest{
{"\u3332", []string{"\u30D5", "\u30A1", "\u30E9", "\u30C3", "\u30C8\u3099", ""}},
// last segment of multi-segment decomposition needs normalization
{"\u3332\u093C", []string{"\u30D5", "\u30A1", "\u30E9", "\u30C3", "\u30C8\u093C\u3099", ""}},
{"\u320E", []string{"\x28", "\uAC00", "\x29"}},
// last segment should be copied to start of buffer.
{"\ufdfa", []string{"\u0635", "\u0644", "\u0649", " ", "\u0627", "\u0644", "\u0644", "\u0647", " ", "\u0639", "\u0644", "\u064a", "\u0647", " ", "\u0648", "\u0633", "\u0644", "\u0645", ""}},
{"\ufdfa" + grave(30), []string{"\u0635", "\u0644", "\u0649", " ", "\u0627", "\u0644", "\u0644", "\u0647", " ", "\u0639", "\u0644", "\u064a", "\u0647", " ", "\u0648", "\u0633", "\u0644", "\u0645" + grave(30), ""}},
{"\uFDFA" + grave(64), []string{"\u0635", "\u0644", "\u0649", " ", "\u0627", "\u0644", "\u0644", "\u0647", " ", "\u0639", "\u0644", "\u064a", "\u0647", " ", "\u0648", "\u0633", "\u0644", "\u0645" + grave(30), cgj + grave(30), cgj + grave(4), ""}},
// Hangul and Jamo are grouped togeter.
{"\uAC00", []string{"\u1100\u1161", ""}},
{"\uAC01", []string{"\u1100\u1161\u11A8", ""}},
{"\u1100\u1161", []string{"\u1100\u1161", ""}},
}
// Note that, by design, segmentation is equal for composing and decomposing forms.
func TestIterSegmentation(t *testing.T) {
segmentTest(t, "SegmentTestD", NFD, segmentTests)
segmentTest(t, "SegmentTestC", NFC, segmentTests)
segmentTest(t, "SegmentTestKD", NFKD, segmentTestsK)
segmentTest(t, "SegmentTestKC", NFKC, segmentTestsK)
}
func segmentTest(t *testing.T, name string, f Form, tests []SegmentTest) {
iter := Iter{}
for i, tt := range tests {
iter.InitString(f, tt.in)
for j, seg := range tt.out {
if seg == "" {
if !iter.Done() {
res := string(iter.Next())
t.Errorf(`%s:%d:%d: expected Done()==true, found segment %+q`, name, i, j, res)
}
continue
}
if iter.Done() {
t.Errorf("%s:%d:%d: Done()==true, want false", name, i, j)
}
seg = f.String(seg)
if res := string(iter.Next()); res != seg {
t.Errorf(`%s:%d:%d" segment was %+q (%d); want %+q (%d)`, name, i, j, pc(res), len(res), pc(seg), len(seg))
}
}
}
}
File diff suppressed because it is too large Load Diff
@@ -1,45 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ignore
// Generate test data for trie code.
package main
import (
"fmt"
)
func main() {
printTestTables()
}
// We take the smallest, largest and an arbitrary value for each
// of the UTF-8 sequence lengths.
var testRunes = []rune{
0x01, 0x0C, 0x7F, // 1-byte sequences
0x80, 0x100, 0x7FF, // 2-byte sequences
0x800, 0x999, 0xFFFF, // 3-byte sequences
0x10000, 0x10101, 0x10FFFF, // 4-byte sequences
0x200, 0x201, 0x202, 0x210, 0x215, // five entries in one sparse block
}
const fileHeader = `// Generated by running
// maketesttables
// DO NOT EDIT
package norm
`
func printTestTables() {
fmt.Print(fileHeader)
fmt.Printf("var testRunes = %#v\n\n", testRunes)
t := newNode()
for i, r := range testRunes {
t.insert(r, uint16(i))
}
t.printTables("testdata")
}
@@ -1,14 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm_test
import (
"testing"
)
func TestPlaceHolder(t *testing.T) {
// Does nothing, just allows the Makefile to be canonical
// while waiting for the package itself to be written.
}
@@ -1,524 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package norm contains types and functions for normalizing Unicode strings.
package norm
import "unicode/utf8"
// A Form denotes a canonical representation of Unicode code points.
// The Unicode-defined normalization and equivalence forms are:
//
// NFC Unicode Normalization Form C
// NFD Unicode Normalization Form D
// NFKC Unicode Normalization Form KC
// NFKD Unicode Normalization Form KD
//
// For a Form f, this documentation uses the notation f(x) to mean
// the bytes or string x converted to the given form.
// A position n in x is called a boundary if conversion to the form can
// proceed independently on both sides:
// f(x) == append(f(x[0:n]), f(x[n:])...)
//
// References: http://unicode.org/reports/tr15/ and
// http://unicode.org/notes/tn5/.
type Form int
const (
NFC Form = iota
NFD
NFKC
NFKD
)
// Bytes returns f(b). May return b if f(b) = b.
func (f Form) Bytes(b []byte) []byte {
src := inputBytes(b)
ft := formTable[f]
n, ok := ft.quickSpan(src, 0, len(b), true)
if ok {
return b
}
out := make([]byte, n, len(b))
copy(out, b[0:n])
rb := reorderBuffer{f: *ft, src: src, nsrc: len(b), out: out, flushF: appendFlush}
return doAppendInner(&rb, n)
}
// String returns f(s).
func (f Form) String(s string) string {
src := inputString(s)
ft := formTable[f]
n, ok := ft.quickSpan(src, 0, len(s), true)
if ok {
return s
}
out := make([]byte, n, len(s))
copy(out, s[0:n])
rb := reorderBuffer{f: *ft, src: src, nsrc: len(s), out: out, flushF: appendFlush}
return string(doAppendInner(&rb, n))
}
// IsNormal returns true if b == f(b).
func (f Form) IsNormal(b []byte) bool {
src := inputBytes(b)
ft := formTable[f]
bp, ok := ft.quickSpan(src, 0, len(b), true)
if ok {
return true
}
rb := reorderBuffer{f: *ft, src: src, nsrc: len(b)}
rb.setFlusher(nil, cmpNormalBytes)
for bp < len(b) {
rb.out = b[bp:]
if bp = decomposeSegment(&rb, bp, true); bp < 0 {
return false
}
bp, _ = rb.f.quickSpan(rb.src, bp, len(b), true)
}
return true
}
func cmpNormalBytes(rb *reorderBuffer) bool {
b := rb.out
for i := 0; i < rb.nrune; i++ {
info := rb.rune[i]
if int(info.size) > len(b) {
return false
}
p := info.pos
pe := p + info.size
for ; p < pe; p++ {
if b[0] != rb.byte[p] {
return false
}
b = b[1:]
}
}
return true
}
// IsNormalString returns true if s == f(s).
func (f Form) IsNormalString(s string) bool {
src := inputString(s)
ft := formTable[f]
bp, ok := ft.quickSpan(src, 0, len(s), true)
if ok {
return true
}
rb := reorderBuffer{f: *ft, src: src, nsrc: len(s)}
rb.setFlusher(nil, func(rb *reorderBuffer) bool {
for i := 0; i < rb.nrune; i++ {
info := rb.rune[i]
if bp+int(info.size) > len(s) {
return false
}
p := info.pos
pe := p + info.size
for ; p < pe; p++ {
if s[bp] != rb.byte[p] {
return false
}
bp++
}
}
return true
})
for bp < len(s) {
if bp = decomposeSegment(&rb, bp, true); bp < 0 {
return false
}
bp, _ = rb.f.quickSpan(rb.src, bp, len(s), true)
}
return true
}
// patchTail fixes a case where a rune may be incorrectly normalized
// if it is followed by illegal continuation bytes. It returns the
// patched buffer and whether the decomposition is still in progress.
func patchTail(rb *reorderBuffer) bool {
info, p := lastRuneStart(&rb.f, rb.out)
if p == -1 || info.size == 0 {
return true
}
end := p + int(info.size)
extra := len(rb.out) - end
if extra > 0 {
// Potentially allocating memory. However, this only
// happens with ill-formed UTF-8.
x := make([]byte, 0)
x = append(x, rb.out[len(rb.out)-extra:]...)
rb.out = rb.out[:end]
decomposeToLastBoundary(rb)
rb.doFlush()
rb.out = append(rb.out, x...)
return false
}
buf := rb.out[p:]
rb.out = rb.out[:p]
decomposeToLastBoundary(rb)
if s := rb.ss.next(info); s == ssStarter {
rb.doFlush()
rb.ss.first(info)
} else if s == ssOverflow {
rb.doFlush()
rb.insertCGJ()
rb.ss = 0
}
rb.insertUnsafe(inputBytes(buf), 0, info)
return true
}
func appendQuick(rb *reorderBuffer, i int) int {
if rb.nsrc == i {
return i
}
end, _ := rb.f.quickSpan(rb.src, i, rb.nsrc, true)
rb.out = rb.src.appendSlice(rb.out, i, end)
return end
}
// Append returns f(append(out, b...)).
// The buffer out must be nil, empty, or equal to f(out).
func (f Form) Append(out []byte, src ...byte) []byte {
return f.doAppend(out, inputBytes(src), len(src))
}
func (f Form) doAppend(out []byte, src input, n int) []byte {
if n == 0 {
return out
}
ft := formTable[f]
// Attempt to do a quickSpan first so we can avoid initializing the reorderBuffer.
if len(out) == 0 {
p, _ := ft.quickSpan(src, 0, n, true)
out = src.appendSlice(out, 0, p)
if p == n {
return out
}
rb := reorderBuffer{f: *ft, src: src, nsrc: n, out: out, flushF: appendFlush}
return doAppendInner(&rb, p)
}
rb := reorderBuffer{f: *ft, src: src, nsrc: n}
return doAppend(&rb, out, 0)
}
func doAppend(rb *reorderBuffer, out []byte, p int) []byte {
rb.setFlusher(out, appendFlush)
src, n := rb.src, rb.nsrc
doMerge := len(out) > 0
if q := src.skipContinuationBytes(p); q > p {
// Move leading non-starters to destination.
rb.out = src.appendSlice(rb.out, p, q)
p = q
doMerge = patchTail(rb)
}
fd := &rb.f
if doMerge {
var info Properties
if p < n {
info = fd.info(src, p)
if !info.BoundaryBefore() || info.nLeadingNonStarters() > 0 {
if p == 0 {
decomposeToLastBoundary(rb)
}
p = decomposeSegment(rb, p, true)
}
}
if info.size == 0 {
rb.doFlush()
// Append incomplete UTF-8 encoding.
return src.appendSlice(rb.out, p, n)
}
if rb.nrune > 0 {
return doAppendInner(rb, p)
}
}
p = appendQuick(rb, p)
return doAppendInner(rb, p)
}
func doAppendInner(rb *reorderBuffer, p int) []byte {
for n := rb.nsrc; p < n; {
p = decomposeSegment(rb, p, true)
p = appendQuick(rb, p)
}
return rb.out
}
// AppendString returns f(append(out, []byte(s))).
// The buffer out must be nil, empty, or equal to f(out).
func (f Form) AppendString(out []byte, src string) []byte {
return f.doAppend(out, inputString(src), len(src))
}
// QuickSpan returns a boundary n such that b[0:n] == f(b[0:n]).
// It is not guaranteed to return the largest such n.
func (f Form) QuickSpan(b []byte) int {
n, _ := formTable[f].quickSpan(inputBytes(b), 0, len(b), true)
return n
}
// quickSpan returns a boundary n such that src[0:n] == f(src[0:n]) and
// whether any non-normalized parts were found. If atEOF is false, n will
// not point past the last segment if this segment might be become
// non-normalized by appending other runes.
func (f *formInfo) quickSpan(src input, i, end int, atEOF bool) (n int, ok bool) {
var lastCC uint8
ss := streamSafe(0)
lastSegStart := i
for n = end; i < n; {
if j := src.skipASCII(i, n); i != j {
i = j
lastSegStart = i - 1
lastCC = 0
ss = 0
continue
}
info := f.info(src, i)
if info.size == 0 {
if atEOF {
// include incomplete runes
return n, true
}
return lastSegStart, true
}
// This block needs to be before the next, because it is possible to
// have an overflow for runes that are starters (e.g. with U+FF9E).
switch ss.next(info) {
case ssStarter:
ss.first(info)
lastSegStart = i
case ssOverflow:
return lastSegStart, false
case ssSuccess:
if lastCC > info.ccc {
return lastSegStart, false
}
}
if f.composing {
if !info.isYesC() {
break
}
} else {
if !info.isYesD() {
break
}
}
lastCC = info.ccc
i += int(info.size)
}
if i == n {
if !atEOF {
n = lastSegStart
}
return n, true
}
return lastSegStart, false
}
// QuickSpanString returns a boundary n such that b[0:n] == f(s[0:n]).
// It is not guaranteed to return the largest such n.
func (f Form) QuickSpanString(s string) int {
n, _ := formTable[f].quickSpan(inputString(s), 0, len(s), true)
return n
}
// FirstBoundary returns the position i of the first boundary in b
// or -1 if b contains no boundary.
func (f Form) FirstBoundary(b []byte) int {
return f.firstBoundary(inputBytes(b), len(b))
}
func (f Form) firstBoundary(src input, nsrc int) int {
i := src.skipContinuationBytes(0)
if i >= nsrc {
return -1
}
fd := formTable[f]
ss := streamSafe(0)
// We should call ss.first here, but we can't as the first rune is
// skipped already. This means FirstBoundary can't really determine
// CGJ insertion points correctly. Luckily it doesn't have to.
// TODO: consider adding NextBoundary
for {
info := fd.info(src, i)
if info.size == 0 {
return -1
}
if s := ss.next(info); s != ssSuccess {
return i
}
i += int(info.size)
if i >= nsrc {
if !info.BoundaryAfter() && !ss.isMax() {
return -1
}
return nsrc
}
}
}
// FirstBoundaryInString returns the position i of the first boundary in s
// or -1 if s contains no boundary.
func (f Form) FirstBoundaryInString(s string) int {
return f.firstBoundary(inputString(s), len(s))
}
// LastBoundary returns the position i of the last boundary in b
// or -1 if b contains no boundary.
func (f Form) LastBoundary(b []byte) int {
return lastBoundary(formTable[f], b)
}
func lastBoundary(fd *formInfo, b []byte) int {
i := len(b)
info, p := lastRuneStart(fd, b)
if p == -1 {
return -1
}
if info.size == 0 { // ends with incomplete rune
if p == 0 { // starts with incomplete rune
return -1
}
i = p
info, p = lastRuneStart(fd, b[:i])
if p == -1 { // incomplete UTF-8 encoding or non-starter bytes without a starter
return i
}
}
if p+int(info.size) != i { // trailing non-starter bytes: illegal UTF-8
return i
}
if info.BoundaryAfter() {
return i
}
ss := streamSafe(0)
v := ss.backwards(info)
for i = p; i >= 0 && v != ssStarter; i = p {
info, p = lastRuneStart(fd, b[:i])
if v = ss.backwards(info); v == ssOverflow {
break
}
if p+int(info.size) != i {
if p == -1 { // no boundary found
return -1
}
return i // boundary after an illegal UTF-8 encoding
}
}
return i
}
// decomposeSegment scans the first segment in src into rb. It inserts 0x034f
// (Grapheme Joiner) when it encounters a sequence of more than 30 non-starters
// and returns the number of bytes consumed from src or iShortDst or iShortSrc.
func decomposeSegment(rb *reorderBuffer, sp int, atEOF bool) int {
// Force one character to be consumed.
info := rb.f.info(rb.src, sp)
if info.size == 0 {
return 0
}
if rb.nrune > 0 {
if s := rb.ss.next(info); s == ssStarter {
goto end
} else if s == ssOverflow {
rb.insertCGJ()
goto end
}
} else {
rb.ss.first(info)
}
if err := rb.insertFlush(rb.src, sp, info); err != iSuccess {
return int(err)
}
for {
sp += int(info.size)
if sp >= rb.nsrc {
if !atEOF && !info.BoundaryAfter() {
return int(iShortSrc)
}
break
}
info = rb.f.info(rb.src, sp)
if info.size == 0 {
if !atEOF {
return int(iShortSrc)
}
break
}
if s := rb.ss.next(info); s == ssStarter {
break
} else if s == ssOverflow {
rb.insertCGJ()
break
}
if err := rb.insertFlush(rb.src, sp, info); err != iSuccess {
return int(err)
}
}
end:
if !rb.doFlush() {
return int(iShortDst)
}
return sp
}
// lastRuneStart returns the runeInfo and position of the last
// rune in buf or the zero runeInfo and -1 if no rune was found.
func lastRuneStart(fd *formInfo, buf []byte) (Properties, int) {
p := len(buf) - 1
for ; p >= 0 && !utf8.RuneStart(buf[p]); p-- {
}
if p < 0 {
return Properties{}, -1
}
return fd.info(inputBytes(buf), p), p
}
// decomposeToLastBoundary finds an open segment at the end of the buffer
// and scans it into rb. Returns the buffer minus the last segment.
func decomposeToLastBoundary(rb *reorderBuffer) {
fd := &rb.f
info, i := lastRuneStart(fd, rb.out)
if int(info.size) != len(rb.out)-i {
// illegal trailing continuation bytes
return
}
if info.BoundaryAfter() {
return
}
var add [maxNonStarters + 1]Properties // stores runeInfo in reverse order
padd := 0
ss := streamSafe(0)
p := len(rb.out)
for {
add[padd] = info
v := ss.backwards(info)
if v == ssOverflow {
// Note that if we have an overflow, it the string we are appending to
// is not correctly normalized. In this case the behavior is undefined.
break
}
padd++
p -= int(info.size)
if v == ssStarter || p < 0 {
break
}
info, i = lastRuneStart(fd, rb.out[:p])
if int(info.size) != p-i {
break
}
}
rb.ss = ss
// Copy bytes for insertion as we may need to overwrite rb.out.
var buf [maxBufferSize * utf8.UTFMax]byte
cp := buf[:copy(buf[:], rb.out[p:])]
rb.out = rb.out[:p]
for padd--; padd >= 0; padd-- {
info = add[padd]
rb.insertUnsafe(inputBytes(cp), 0, info)
cp = cp[info.size:]
}
}
File diff suppressed because it is too large Load Diff
@@ -1,318 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ignore
package main
import (
"bufio"
"bytes"
"flag"
"fmt"
"log"
"net/http"
"os"
"path"
"regexp"
"runtime"
"strconv"
"strings"
"time"
"unicode"
"unicode/utf8"
"code.google.com/p/go.text/unicode/norm"
)
func main() {
flag.Parse()
loadTestData()
CharacterByCharacterTests()
StandardTests()
PerformanceTest()
if errorCount == 0 {
fmt.Println("PASS")
}
}
const file = "NormalizationTest.txt"
var url = flag.String("url",
"http://www.unicode.org/Public/"+unicode.Version+"/ucd/"+file,
"URL of Unicode database directory")
var localFiles = flag.Bool("local",
false,
"data files have been copied to the current directory; for debugging only")
var logger = log.New(os.Stderr, "", log.Lshortfile)
// This regression test runs the test set in NormalizationTest.txt
// (taken from http://www.unicode.org/Public/<unicode.Version>/ucd/).
//
// NormalizationTest.txt has form:
// @Part0 # Specific cases
// #
// 1E0A;1E0A;0044 0307;1E0A;0044 0307; # (Ḋ; Ḋ; D◌̇; Ḋ; D◌̇; ) LATIN CAPITAL LETTER D WITH DOT ABOVE
// 1E0C;1E0C;0044 0323;1E0C;0044 0323; # (Ḍ; Ḍ; D◌̣; Ḍ; D◌̣; ) LATIN CAPITAL LETTER D WITH DOT BELOW
//
// Each test has 5 columns (c1, c2, c3, c4, c5), where
// (c1, c2, c3, c4, c5) == (c1, NFC(c1), NFD(c1), NFKC(c1), NFKD(c1))
//
// CONFORMANCE:
// 1. The following invariants must be true for all conformant implementations
//
// NFC
// c2 == NFC(c1) == NFC(c2) == NFC(c3)
// c4 == NFC(c4) == NFC(c5)
//
// NFD
// c3 == NFD(c1) == NFD(c2) == NFD(c3)
// c5 == NFD(c4) == NFD(c5)
//
// NFKC
// c4 == NFKC(c1) == NFKC(c2) == NFKC(c3) == NFKC(c4) == NFKC(c5)
//
// NFKD
// c5 == NFKD(c1) == NFKD(c2) == NFKD(c3) == NFKD(c4) == NFKD(c5)
//
// 2. For every code point X assigned in this version of Unicode that is not
// specifically listed in Part 1, the following invariants must be true
// for all conformant implementations:
//
// X == NFC(X) == NFD(X) == NFKC(X) == NFKD(X)
//
// Column types.
const (
cRaw = iota
cNFC
cNFD
cNFKC
cNFKD
cMaxColumns
)
// Holds data from NormalizationTest.txt
var part []Part
type Part struct {
name string
number int
tests []Test
}
type Test struct {
name string
partnr int
number int
r rune // used for character by character test
cols [cMaxColumns]string // Each has 5 entries, see below.
}
func (t Test) Name() string {
if t.number < 0 {
return part[t.partnr].name
}
return fmt.Sprintf("%s:%d", part[t.partnr].name, t.number)
}
var partRe = regexp.MustCompile(`@Part(\d) # (.*)$`)
var testRe = regexp.MustCompile(`^` + strings.Repeat(`([\dA-F ]+);`, 5) + ` # (.*)$`)
var counter int
// Load the data form NormalizationTest.txt
func loadTestData() {
if *localFiles {
pwd, _ := os.Getwd()
*url = "file://" + path.Join(pwd, file)
}
t := &http.Transport{}
t.RegisterProtocol("file", http.NewFileTransport(http.Dir("/")))
c := &http.Client{Transport: t}
resp, err := c.Get(*url)
if err != nil {
logger.Fatal(err)
}
if resp.StatusCode != 200 {
logger.Fatal("bad GET status for "+file, resp.Status)
}
f := resp.Body
defer f.Close()
scanner := bufio.NewScanner(f)
for scanner.Scan() {
line := scanner.Text()
if len(line) == 0 || line[0] == '#' {
continue
}
m := partRe.FindStringSubmatch(line)
if m != nil {
if len(m) < 3 {
logger.Fatal("Failed to parse Part: ", line)
}
i, err := strconv.Atoi(m[1])
if err != nil {
logger.Fatal(err)
}
name := m[2]
part = append(part, Part{name: name[:len(name)-1], number: i})
continue
}
m = testRe.FindStringSubmatch(line)
if m == nil || len(m) < 7 {
logger.Fatalf(`Failed to parse: "%s" result: %#v`, line, m)
}
test := Test{name: m[6], partnr: len(part) - 1, number: counter}
counter++
for j := 1; j < len(m)-1; j++ {
for _, split := range strings.Split(m[j], " ") {
r, err := strconv.ParseUint(split, 16, 64)
if err != nil {
logger.Fatal(err)
}
if test.r == 0 {
// save for CharacterByCharacterTests
test.r = rune(r)
}
var buf [utf8.UTFMax]byte
sz := utf8.EncodeRune(buf[:], rune(r))
test.cols[j-1] += string(buf[:sz])
}
}
part := &part[len(part)-1]
part.tests = append(part.tests, test)
}
if scanner.Err() != nil {
logger.Fatal(scanner.Err())
}
}
var fstr = []string{"NFC", "NFD", "NFKC", "NFKD"}
var errorCount int
func cmpResult(t *Test, name string, f norm.Form, gold, test, result string) {
if gold != result {
errorCount++
if errorCount > 20 {
return
}
logger.Printf("%s:%s: %s(%+q)=%+q; want %+q: %s",
t.Name(), name, fstr[f], test, result, gold, t.name)
}
}
func cmpIsNormal(t *Test, name string, f norm.Form, test string, result, want bool) {
if result != want {
errorCount++
if errorCount > 20 {
return
}
logger.Printf("%s:%s: %s(%+q)=%v; want %v", t.Name(), name, fstr[f], test, result, want)
}
}
func doTest(t *Test, f norm.Form, gold, test string) {
testb := []byte(test)
result := f.Bytes(testb)
cmpResult(t, "Bytes", f, gold, test, string(result))
sresult := f.String(test)
cmpResult(t, "String", f, gold, test, sresult)
acc := []byte{}
i := norm.Iter{}
i.InitString(f, test)
for !i.Done() {
acc = append(acc, i.Next()...)
}
cmpResult(t, "Iter.Next", f, gold, test, string(acc))
buf := make([]byte, 128)
acc = nil
for p := 0; p < len(testb); {
nDst, nSrc, _ := f.Transform(buf, testb[p:], true)
acc = append(acc, buf[:nDst]...)
p += nSrc
}
cmpResult(t, "Transform", f, gold, test, string(acc))
for i := range test {
out := f.Append(f.Bytes([]byte(test[:i])), []byte(test[i:])...)
cmpResult(t, fmt.Sprintf(":Append:%d", i), f, gold, test, string(out))
}
cmpIsNormal(t, "IsNormal", f, test, f.IsNormal([]byte(test)), test == gold)
cmpIsNormal(t, "IsNormalString", f, test, f.IsNormalString(test), test == gold)
}
func doConformanceTests(t *Test, partn int) {
for i := 0; i <= 2; i++ {
doTest(t, norm.NFC, t.cols[1], t.cols[i])
doTest(t, norm.NFD, t.cols[2], t.cols[i])
doTest(t, norm.NFKC, t.cols[3], t.cols[i])
doTest(t, norm.NFKD, t.cols[4], t.cols[i])
}
for i := 3; i <= 4; i++ {
doTest(t, norm.NFC, t.cols[3], t.cols[i])
doTest(t, norm.NFD, t.cols[4], t.cols[i])
doTest(t, norm.NFKC, t.cols[3], t.cols[i])
doTest(t, norm.NFKD, t.cols[4], t.cols[i])
}
}
func CharacterByCharacterTests() {
tests := part[1].tests
var last rune = 0
for i := 0; i <= len(tests); i++ { // last one is special case
var r rune
if i == len(tests) {
r = 0x2FA1E // Don't have to go to 0x10FFFF
} else {
r = tests[i].r
}
for last++; last < r; last++ {
// Check all characters that were not explicitly listed in the test.
t := &Test{partnr: 1, number: -1}
char := string(last)
doTest(t, norm.NFC, char, char)
doTest(t, norm.NFD, char, char)
doTest(t, norm.NFKC, char, char)
doTest(t, norm.NFKD, char, char)
}
if i < len(tests) {
doConformanceTests(&tests[i], 1)
}
}
}
func StandardTests() {
for _, j := range []int{0, 2, 3} {
for _, test := range part[j].tests {
doConformanceTests(&test, j)
}
}
}
// PerformanceTest verifies that normalization is O(n). If any of the
// code does not properly check for maxCombiningChars, normalization
// may exhibit O(n**2) behavior.
func PerformanceTest() {
runtime.GOMAXPROCS(2)
success := make(chan bool, 1)
go func() {
buf := bytes.Repeat([]byte("\u035D"), 1024*1024)
buf = append(buf, "\u035B"...)
norm.NFC.Append(nil, buf...)
success <- true
}()
timeout := time.After(1 * time.Second)
select {
case <-success:
// test completed before the timeout
case <-timeout:
errorCount++
logger.Printf(`unexpectedly long time to complete PerformanceTest`)
}
}
@@ -1,126 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import "io"
type normWriter struct {
rb reorderBuffer
w io.Writer
buf []byte
}
// Write implements the standard write interface. If the last characters are
// not at a normalization boundary, the bytes will be buffered for the next
// write. The remaining bytes will be written on close.
func (w *normWriter) Write(data []byte) (n int, err error) {
// Process data in pieces to keep w.buf size bounded.
const chunk = 4000
for len(data) > 0 {
// Normalize into w.buf.
m := len(data)
if m > chunk {
m = chunk
}
w.rb.src = inputBytes(data[:m])
w.rb.nsrc = m
w.buf = doAppend(&w.rb, w.buf, 0)
data = data[m:]
n += m
// Write out complete prefix, save remainder.
// Note that lastBoundary looks back at most 31 runes.
i := lastBoundary(&w.rb.f, w.buf)
if i == -1 {
i = 0
}
if i > 0 {
if _, err = w.w.Write(w.buf[:i]); err != nil {
break
}
bn := copy(w.buf, w.buf[i:])
w.buf = w.buf[:bn]
}
}
return n, err
}
// Close forces data that remains in the buffer to be written.
func (w *normWriter) Close() error {
if len(w.buf) > 0 {
_, err := w.w.Write(w.buf)
if err != nil {
return err
}
}
return nil
}
// Writer returns a new writer that implements Write(b)
// by writing f(b) to w. The returned writer may use an
// an internal buffer to maintain state across Write calls.
// Calling its Close method writes any buffered data to w.
func (f Form) Writer(w io.Writer) io.WriteCloser {
wr := &normWriter{rb: reorderBuffer{}, w: w}
wr.rb.init(f, nil)
return wr
}
type normReader struct {
rb reorderBuffer
r io.Reader
inbuf []byte
outbuf []byte
bufStart int
lastBoundary int
err error
}
// Read implements the standard read interface.
func (r *normReader) Read(p []byte) (int, error) {
for {
if r.lastBoundary-r.bufStart > 0 {
n := copy(p, r.outbuf[r.bufStart:r.lastBoundary])
r.bufStart += n
if r.lastBoundary-r.bufStart > 0 {
return n, nil
}
return n, r.err
}
if r.err != nil {
return 0, r.err
}
outn := copy(r.outbuf, r.outbuf[r.lastBoundary:])
r.outbuf = r.outbuf[0:outn]
r.bufStart = 0
n, err := r.r.Read(r.inbuf)
r.rb.src = inputBytes(r.inbuf[0:n])
r.rb.nsrc, r.err = n, err
if n > 0 {
r.outbuf = doAppend(&r.rb, r.outbuf, 0)
}
if err == io.EOF {
r.lastBoundary = len(r.outbuf)
} else {
r.lastBoundary = lastBoundary(&r.rb.f, r.outbuf)
if r.lastBoundary == -1 {
r.lastBoundary = 0
}
}
}
panic("should not reach here")
}
// Reader returns a new reader that implements Read
// by reading data from r and returning f(data).
func (f Form) Reader(r io.Reader) io.Reader {
const chunk = 4000
buf := make([]byte, chunk)
rr := &normReader{rb: reorderBuffer{}, r: r, inbuf: buf}
rr.rb.init(f, buf)
return rr
}
@@ -1,56 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import (
"bytes"
"fmt"
"testing"
)
var bufSizes = []int{1, 2, 3, 4, 5, 6, 7, 8, 100, 101, 102, 103, 4000, 4001, 4002, 4003}
func readFunc(size int) appendFunc {
return func(f Form, out []byte, s string) []byte {
out = append(out, s...)
r := f.Reader(bytes.NewBuffer(out))
buf := make([]byte, size)
result := []byte{}
for n, err := 0, error(nil); err == nil; {
n, err = r.Read(buf)
result = append(result, buf[:n]...)
}
return result
}
}
func TestReader(t *testing.T) {
for _, s := range bufSizes {
name := fmt.Sprintf("TestReader%d", s)
runNormTests(t, name, readFunc(s))
}
}
func writeFunc(size int) appendFunc {
return func(f Form, out []byte, s string) []byte {
in := append(out, s...)
result := new(bytes.Buffer)
w := f.Writer(result)
buf := make([]byte, size)
for n := 0; len(in) > 0; in = in[n:] {
n = copy(buf, in)
_, _ = w.Write(buf[:n])
}
w.Close()
return result.Bytes()
}
}
func TestWriter(t *testing.T) {
for _, s := range bufSizes {
name := fmt.Sprintf("TestWriter%d", s)
runNormTests(t, name, writeFunc(s))
}
}
File diff suppressed because it is too large Load Diff
@@ -1,85 +0,0 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import (
"unicode/utf8"
"code.google.com/p/go.text/transform"
)
// Transform implements the transform.Transformer interface. It may need to
// write segments of up to MaxSegmentSize at once. Users should either catch
// ErrShortDst and allow dst to grow or have dst be at least of size
// MaxTransformChunkSize to be guaranteed of progress.
func (f Form) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
n := 0
// Cap the maximum number of src bytes to check.
b := src
eof := atEOF
if ns := len(dst); ns < len(b) {
err = transform.ErrShortDst
eof = false
b = b[:ns]
}
i, ok := formTable[f].quickSpan(inputBytes(b), n, len(b), eof)
n += copy(dst[n:], b[n:i])
if !ok {
nDst, nSrc, err = f.transform(dst[n:], src[n:], atEOF)
return nDst + n, nSrc + n, err
}
if n < len(src) && !atEOF {
err = transform.ErrShortSrc
}
return n, n, err
}
func flushTransform(rb *reorderBuffer) bool {
// Write out (must fully fit in dst, or else it is a ErrShortDst).
if len(rb.out) < rb.nrune*utf8.UTFMax {
return false
}
rb.out = rb.out[rb.flushCopy(rb.out):]
return true
}
var errs = []error{nil, transform.ErrShortDst, transform.ErrShortSrc}
// transform implements the transform.Transformer interface. It is only called
// when quickSpan does not pass for a given string.
func (f Form) transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
// TODO: get rid of reorderBuffer. See CL 23460044.
rb := reorderBuffer{}
rb.init(f, src)
for {
// Load segment into reorder buffer.
rb.setFlusher(dst[nDst:], flushTransform)
end := decomposeSegment(&rb, nSrc, atEOF)
if end < 0 {
return nDst, nSrc, errs[-end]
}
nDst = len(dst) - len(rb.out)
nSrc = end
// Next quickSpan.
end = rb.nsrc
eof := atEOF
if n := nSrc + len(dst) - nDst; n < end {
err = transform.ErrShortDst
end = n
eof = false
}
end, ok := rb.f.quickSpan(rb.src, nSrc, end, eof)
n := copy(dst[nDst:], rb.src.bytes[nSrc:end])
nSrc += n
nDst += n
if ok {
if n < rb.nsrc && !atEOF {
err = transform.ErrShortSrc
}
return nDst, nSrc, err
}
}
}
@@ -1,101 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import (
"fmt"
"testing"
"code.google.com/p/go.text/transform"
)
func TestTransform(t *testing.T) {
tests := []struct {
f Form
in, out string
eof bool
dstSize int
err error
}{
{NFC, "ab", "ab", true, 2, nil},
{NFC, "qx", "qx", true, 2, nil},
{NFD, "qx", "qx", true, 2, nil},
{NFC, "", "", true, 1, nil},
{NFD, "", "", true, 1, nil},
{NFC, "", "", false, 1, nil},
{NFD, "", "", false, 1, nil},
// Normalized segment does not fit in destination.
{NFD, "ö", "", true, 1, transform.ErrShortDst},
{NFD, "ö", "", true, 2, transform.ErrShortDst},
// As an artifact of the algorithm, only full segments are written.
// This is not strictly required, and some bytes could be written.
// In practice, for Transform to not block, the destination buffer
// should be at least MaxSegmentSize to work anyway and these edge
// conditions will be relatively rare.
{NFC, "ab", "", true, 1, transform.ErrShortDst},
// This is even true for inert runes.
{NFC, "qx", "", true, 1, transform.ErrShortDst},
{NFC, "a\u0300abc", "\u00e0a", true, 4, transform.ErrShortDst},
// We cannot write a segment if succesive runes could still change the result.
{NFD, "ö", "", false, 3, transform.ErrShortSrc},
{NFC, "a\u0300", "", false, 4, transform.ErrShortSrc},
{NFD, "a\u0300", "", false, 4, transform.ErrShortSrc},
{NFC, "ö", "", false, 3, transform.ErrShortSrc},
{NFC, "a\u0300", "", true, 1, transform.ErrShortDst},
// Theoretically could fit, but won't due to simplified checks.
{NFC, "a\u0300", "", true, 2, transform.ErrShortDst},
{NFC, "a\u0300", "", true, 3, transform.ErrShortDst},
{NFC, "a\u0300", "\u00e0", true, 4, nil},
{NFD, "öa\u0300", "o\u0308", false, 8, transform.ErrShortSrc},
{NFD, "öa\u0300ö", "o\u0308a\u0300", true, 8, transform.ErrShortDst},
{NFD, "öa\u0300ö", "o\u0308a\u0300", false, 12, transform.ErrShortSrc},
// Illegal input is copied verbatim.
{NFD, "\xbd\xb2=\xbc ", "\xbd\xb2=\xbc ", true, 8, nil},
}
b := make([]byte, 100)
for i, tt := range tests {
nDst, _, err := tt.f.Transform(b[:tt.dstSize], []byte(tt.in), tt.eof)
out := string(b[:nDst])
if out != tt.out || err != tt.err {
t.Errorf("%d: was %+q (%v); want %+q (%v)", i, out, err, tt.out, tt.err)
}
if want := tt.f.String(tt.in)[:nDst]; want != out {
t.Errorf("%d: incorect normalization: was %+q; want %+q", i, out, want)
}
}
}
var transBufSizes = []int{
MaxTransformChunkSize,
3 * MaxTransformChunkSize / 2,
2 * MaxTransformChunkSize,
3 * MaxTransformChunkSize,
100 * MaxTransformChunkSize,
}
func doTransNorm(f Form, buf []byte, b []byte) []byte {
acc := []byte{}
for p := 0; p < len(b); {
nd, ns, _ := f.Transform(buf[:], b[p:], true)
p += ns
acc = append(acc, buf[:nd]...)
}
return acc
}
func TestTransformNorm(t *testing.T) {
for _, sz := range transBufSizes {
buf := make([]byte, sz)
runNormTests(t, fmt.Sprintf("Transform:%d", sz), func(f Form, out []byte, s string) []byte {
return doTransNorm(f, buf, append(out, s...))
})
}
}
-232
View File
@@ -1,232 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
type valueRange struct {
value uint16 // header: value:stride
lo, hi byte // header: lo:n
}
type trie struct {
index []uint8
values []uint16
sparse []valueRange
sparseOffset []uint16
cutoff uint8 // indices >= cutoff are sparse
}
// lookupValue determines the type of block n and looks up the value for b.
// For n < t.cutoff, the block is a simple lookup table. Otherwise, the block
// is a list of ranges with an accompanying value. Given a matching range r,
// the value for b is by r.value + (b - r.lo) * stride.
func (t *trie) lookupValue(n uint8, b byte) uint16 {
if n < t.cutoff {
return t.values[uint16(n)<<6+uint16(b)]
}
offset := t.sparseOffset[n-t.cutoff]
header := t.sparse[offset]
lo := offset + 1
hi := lo + uint16(header.lo)
for lo < hi {
m := lo + (hi-lo)/2
r := t.sparse[m]
if r.lo <= b && b <= r.hi {
return r.value + uint16(b-r.lo)*header.value
}
if b < r.lo {
hi = m
} else {
lo = m + 1
}
}
return 0
}
const (
t1 = 0x00 // 0000 0000
tx = 0x80 // 1000 0000
t2 = 0xC0 // 1100 0000
t3 = 0xE0 // 1110 0000
t4 = 0xF0 // 1111 0000
t5 = 0xF8 // 1111 1000
t6 = 0xFC // 1111 1100
te = 0xFE // 1111 1110
)
// lookup returns the trie value for the first UTF-8 encoding in s and
// the width in bytes of this encoding. The size will be 0 if s does not
// hold enough bytes to complete the encoding. len(s) must be greater than 0.
func (t *trie) lookup(s []byte) (v uint16, sz int) {
c0 := s[0]
switch {
case c0 < tx:
return t.values[c0], 1
case c0 < t2:
return 0, 1
case c0 < t3:
if len(s) < 2 {
return 0, 0
}
i := t.index[c0]
c1 := s[1]
if c1 < tx || t2 <= c1 {
return 0, 1
}
return t.lookupValue(i, c1), 2
case c0 < t4:
if len(s) < 3 {
return 0, 0
}
i := t.index[c0]
c1 := s[1]
if c1 < tx || t2 <= c1 {
return 0, 1
}
o := uint16(i)<<6 + uint16(c1)
i = t.index[o]
c2 := s[2]
if c2 < tx || t2 <= c2 {
return 0, 2
}
return t.lookupValue(i, c2), 3
case c0 < t5:
if len(s) < 4 {
return 0, 0
}
i := t.index[c0]
c1 := s[1]
if c1 < tx || t2 <= c1 {
return 0, 1
}
o := uint16(i)<<6 + uint16(c1)
i = t.index[o]
c2 := s[2]
if c2 < tx || t2 <= c2 {
return 0, 2
}
o = uint16(i)<<6 + uint16(c2)
i = t.index[o]
c3 := s[3]
if c3 < tx || t2 <= c3 {
return 0, 3
}
return t.lookupValue(i, c3), 4
}
// Illegal rune
return 0, 1
}
// lookupString returns the trie value for the first UTF-8 encoding in s and
// the width in bytes of this encoding. The size will be 0 if s does not
// hold enough bytes to complete the encoding. len(s) must be greater than 0.
func (t *trie) lookupString(s string) (v uint16, sz int) {
c0 := s[0]
switch {
case c0 < tx:
return t.values[c0], 1
case c0 < t2:
return 0, 1
case c0 < t3:
if len(s) < 2 {
return 0, 0
}
i := t.index[c0]
c1 := s[1]
if c1 < tx || t2 <= c1 {
return 0, 1
}
return t.lookupValue(i, c1), 2
case c0 < t4:
if len(s) < 3 {
return 0, 0
}
i := t.index[c0]
c1 := s[1]
if c1 < tx || t2 <= c1 {
return 0, 1
}
o := uint16(i)<<6 + uint16(c1)
i = t.index[o]
c2 := s[2]
if c2 < tx || t2 <= c2 {
return 0, 2
}
return t.lookupValue(i, c2), 3
case c0 < t5:
if len(s) < 4 {
return 0, 0
}
i := t.index[c0]
c1 := s[1]
if c1 < tx || t2 <= c1 {
return 0, 1
}
o := uint16(i)<<6 + uint16(c1)
i = t.index[o]
c2 := s[2]
if c2 < tx || t2 <= c2 {
return 0, 2
}
o = uint16(i)<<6 + uint16(c2)
i = t.index[o]
c3 := s[3]
if c3 < tx || t2 <= c3 {
return 0, 3
}
return t.lookupValue(i, c3), 4
}
// Illegal rune
return 0, 1
}
// lookupUnsafe returns the trie value for the first UTF-8 encoding in s.
// s must hold a full encoding.
func (t *trie) lookupUnsafe(s []byte) uint16 {
c0 := s[0]
if c0 < tx {
return t.values[c0]
}
if c0 < t2 {
return 0
}
i := t.index[c0]
if c0 < t3 {
return t.lookupValue(i, s[1])
}
i = t.index[uint16(i)<<6+uint16(s[1])]
if c0 < t4 {
return t.lookupValue(i, s[2])
}
i = t.index[uint16(i)<<6+uint16(s[2])]
if c0 < t5 {
return t.lookupValue(i, s[3])
}
return 0
}
// lookupStringUnsafe returns the trie value for the first UTF-8 encoding in s.
// s must hold a full encoding.
func (t *trie) lookupStringUnsafe(s string) uint16 {
c0 := s[0]
if c0 < tx {
return t.values[c0]
}
if c0 < t2 {
return 0
}
i := t.index[c0]
if c0 < t3 {
return t.lookupValue(i, s[1])
}
i = t.index[uint16(i)<<6+uint16(s[1])]
if c0 < t4 {
return t.lookupValue(i, s[2])
}
i = t.index[uint16(i)<<6+uint16(s[2])]
if c0 < t5 {
return t.lookupValue(i, s[3])
}
return 0
}
@@ -1,152 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import (
"testing"
"unicode/utf8"
)
// Test data is located in triedata_test.go; generated by maketesttables.
var testdata = testdataTrie
type rangeTest struct {
block uint8
lookup byte
result uint16
table []valueRange
offsets []uint16
}
var range1Off = []uint16{0, 2}
var range1 = []valueRange{
{0, 1, 0},
{1, 0x80, 0x80},
{0, 2, 0},
{1, 0x80, 0x80},
{9, 0xff, 0xff},
}
var rangeTests = []rangeTest{
{10, 0x80, 1, range1, range1Off},
{10, 0x00, 0, range1, range1Off},
{11, 0x80, 1, range1, range1Off},
{11, 0xff, 9, range1, range1Off},
{11, 0x00, 0, range1, range1Off},
}
func TestLookupSparse(t *testing.T) {
for i, test := range rangeTests {
n := trie{sparse: test.table, sparseOffset: test.offsets, cutoff: 10}
v := n.lookupValue(test.block, test.lookup)
if v != test.result {
t.Errorf("LookupSparse:%d: found %X; want %X", i, v, test.result)
}
}
}
// Test cases for illegal runes.
type trietest struct {
size int
bytes []byte
}
var tests = []trietest{
// illegal runes
{1, []byte{0x80}},
{1, []byte{0xFF}},
{1, []byte{t2, tx - 1}},
{1, []byte{t2, t2}},
{2, []byte{t3, tx, tx - 1}},
{2, []byte{t3, tx, t2}},
{1, []byte{t3, tx - 1, tx}},
{3, []byte{t4, tx, tx, tx - 1}},
{3, []byte{t4, tx, tx, t2}},
{1, []byte{t4, t2, tx, tx - 1}},
{2, []byte{t4, tx, t2, tx - 1}},
// short runes
{0, []byte{t2}},
{0, []byte{t3, tx}},
{0, []byte{t4, tx, tx}},
// we only support UTF-8 up to utf8.UTFMax bytes (4 bytes)
{1, []byte{t5, tx, tx, tx, tx}},
{1, []byte{t6, tx, tx, tx, tx, tx}},
}
func mkUTF8(r rune) ([]byte, int) {
var b [utf8.UTFMax]byte
sz := utf8.EncodeRune(b[:], r)
return b[:sz], sz
}
func TestLookup(t *testing.T) {
for i, tt := range testRunes {
b, szg := mkUTF8(tt)
v, szt := testdata.lookup(b)
if int(v) != i {
t.Errorf("lookup(%U): found value %#x, expected %#x", tt, v, i)
}
if szt != szg {
t.Errorf("lookup(%U): found size %d, expected %d", tt, szt, szg)
}
}
for i, tt := range tests {
v, sz := testdata.lookup(tt.bytes)
if v != 0 {
t.Errorf("lookup of illegal rune, case %d: found value %#x, expected 0", i, v)
}
if sz != tt.size {
t.Errorf("lookup of illegal rune, case %d: found size %d, expected %d", i, sz, tt.size)
}
}
// Verify defaults.
if v, _ := testdata.lookup([]byte{0xC1, 0x8C}); v != 0 {
t.Errorf("lookup of non-existing rune should be 0; found %X", v)
}
}
func TestLookupUnsafe(t *testing.T) {
for i, tt := range testRunes {
b, _ := mkUTF8(tt)
v := testdata.lookupUnsafe(b)
if int(v) != i {
t.Errorf("lookupUnsafe(%U): found value %#x, expected %#x", i, v, i)
}
}
}
func TestLookupString(t *testing.T) {
for i, tt := range testRunes {
b, szg := mkUTF8(tt)
v, szt := testdata.lookupString(string(b))
if int(v) != i {
t.Errorf("lookup(%U): found value %#x, expected %#x", i, v, i)
}
if szt != szg {
t.Errorf("lookup(%U): found size %d, expected %d", i, szt, szg)
}
}
for i, tt := range tests {
v, sz := testdata.lookupString(string(tt.bytes))
if int(v) != 0 {
t.Errorf("lookup of illegal rune, case %d: found value %#x, expected 0", i, v)
}
if sz != tt.size {
t.Errorf("lookup of illegal rune, case %d: found size %d, expected %d", i, sz, tt.size)
}
}
}
func TestLookupStringUnsafe(t *testing.T) {
for i, tt := range testRunes {
b, _ := mkUTF8(tt)
v := testdata.lookupStringUnsafe(string(b))
if int(v) != i {
t.Errorf("lookupUnsafe(%U): found value %#x, expected %#x", i, v, i)
}
}
}
@@ -1,85 +0,0 @@
// Generated by running
// maketesttables
// DO NOT EDIT
package norm
var testRunes = []int32{1, 12, 127, 128, 256, 2047, 2048, 2457, 65535, 65536, 65793, 1114111, 512, 513, 514, 528, 533}
// testdataValues: 192 entries, 384 bytes
// Block 2 is the null block.
var testdataValues = [192]uint16{
// Block 0x0, offset 0x0
0x000c: 0x0001,
// Block 0x1, offset 0x40
0x007f: 0x0002,
// Block 0x2, offset 0x80
}
// testdataSparseOffset: 10 entries, 20 bytes
var testdataSparseOffset = []uint16{0x0, 0x2, 0x4, 0x8, 0xa, 0xc, 0xe, 0x10, 0x12, 0x14}
// testdataSparseValues: 22 entries, 88 bytes
var testdataSparseValues = [22]valueRange{
// Block 0x0, offset 0x1
{value: 0x0000, lo: 0x01},
{value: 0x0003, lo: 0x80, hi: 0x80},
// Block 0x1, offset 0x2
{value: 0x0000, lo: 0x01},
{value: 0x0004, lo: 0x80, hi: 0x80},
// Block 0x2, offset 0x3
{value: 0x0001, lo: 0x03},
{value: 0x000c, lo: 0x80, hi: 0x82},
{value: 0x000f, lo: 0x90, hi: 0x90},
{value: 0x0010, lo: 0x95, hi: 0x95},
// Block 0x3, offset 0x4
{value: 0x0000, lo: 0x01},
{value: 0x0005, lo: 0xbf, hi: 0xbf},
// Block 0x4, offset 0x5
{value: 0x0000, lo: 0x01},
{value: 0x0006, lo: 0x80, hi: 0x80},
// Block 0x5, offset 0x6
{value: 0x0000, lo: 0x01},
{value: 0x0007, lo: 0x99, hi: 0x99},
// Block 0x6, offset 0x7
{value: 0x0000, lo: 0x01},
{value: 0x0008, lo: 0xbf, hi: 0xbf},
// Block 0x7, offset 0x8
{value: 0x0000, lo: 0x01},
{value: 0x0009, lo: 0x80, hi: 0x80},
// Block 0x8, offset 0x9
{value: 0x0000, lo: 0x01},
{value: 0x000a, lo: 0x81, hi: 0x81},
// Block 0x9, offset 0xa
{value: 0x0000, lo: 0x01},
{value: 0x000b, lo: 0xbf, hi: 0xbf},
}
// testdataLookup: 640 bytes
// Block 0 is the null block.
var testdataLookup = [640]uint8{
// Block 0x0, offset 0x0
// Block 0x1, offset 0x40
// Block 0x2, offset 0x80
// Block 0x3, offset 0xc0
0x0c2: 0x01, 0x0c4: 0x02,
0x0c8: 0x03,
0x0df: 0x04,
0x0e0: 0x02,
0x0ef: 0x03,
0x0f0: 0x05, 0x0f4: 0x07,
// Block 0x4, offset 0x100
0x120: 0x05, 0x126: 0x06,
// Block 0x5, offset 0x140
0x17f: 0x07,
// Block 0x6, offset 0x180
0x180: 0x08, 0x184: 0x09,
// Block 0x7, offset 0x1c0
0x1d0: 0x04,
// Block 0x8, offset 0x200
0x23f: 0x0a,
// Block 0x9, offset 0x240
0x24f: 0x06,
}
var testdataTrie = trie{testdataLookup[:], testdataValues[:], testdataSparseValues[:], testdataSparseOffset[:], 1}
-317
View File
@@ -1,317 +0,0 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ignore
// Trie table generator.
// Used by make*tables tools to generate a go file with trie data structures
// for mapping UTF-8 to a 16-bit value. All but the last byte in a UTF-8 byte
// sequence are used to lookup offsets in the index table to be used for the
// next byte. The last byte is used to index into a table with 16-bit values.
package main
import (
"fmt"
"hash/crc32"
"log"
"unicode/utf8"
)
const (
blockSize = 64
blockOffset = 2 // Subtract two blocks to compensate for the 0x80 added to continuation bytes.
maxSparseEntries = 16
)
// Intermediate trie structure
type trieNode struct {
table [256]*trieNode
value int
b byte
leaf bool
}
func newNode() *trieNode {
return new(trieNode)
}
func (n trieNode) String() string {
s := fmt.Sprint("trieNode{table: { non-nil at index: ")
for i, v := range n.table {
if v != nil {
s += fmt.Sprintf("%d, ", i)
}
}
s += fmt.Sprintf("}, value:%#x, b:%#x leaf:%v}", n.value, n.b, n.leaf)
return s
}
func (n trieNode) isInternal() bool {
internal := true
for i := 0; i < 256; i++ {
if nn := n.table[i]; nn != nil {
if !internal && !nn.leaf {
log.Fatalf("triegen: isInternal: node contains both leaf and non-leaf children (%v)", n)
}
internal = internal && !nn.leaf
}
}
return internal
}
func (n trieNode) mostFrequentStride() int {
counts := make(map[int]int)
v := 0
for _, t := range n.table[0x80 : 0x80+blockSize] {
if t != nil {
if stride := t.value - v; v != 0 && stride >= 0 {
counts[stride]++
}
v = t.value
} else {
v = 0
}
}
var maxs, maxc int
for stride, cnt := range counts {
if cnt > maxc || (cnt == maxc && stride < maxs) {
maxs, maxc = stride, cnt
}
}
return maxs
}
func (n trieNode) countSparseEntries() int {
stride := n.mostFrequentStride()
var count, v int
for _, t := range n.table[0x80 : 0x80+blockSize] {
tv := 0
if t != nil {
tv = t.value
}
if tv-v != stride {
if tv != 0 {
count++
}
}
v = tv
}
return count
}
func (n *trieNode) insert(r rune, value uint16) {
var p [utf8.UTFMax]byte
sz := utf8.EncodeRune(p[:], r)
for i := 0; i < sz; i++ {
if n.leaf {
log.Fatalf("triegen: insert: node (%#v) should not be a leaf", n)
}
nn := n.table[p[i]]
if nn == nil {
nn = newNode()
nn.b = p[i]
n.table[p[i]] = nn
}
n = nn
}
n.value = int(value)
n.leaf = true
}
type nodeIndex struct {
lookupBlocks []*trieNode
valueBlocks []*trieNode
sparseBlocks []*trieNode
sparseOffset []uint16
sparseCount int
lookupBlockIdx map[uint32]int
valueBlockIdx map[uint32]int
}
func newIndex() *nodeIndex {
index := &nodeIndex{}
index.lookupBlocks = make([]*trieNode, 0)
index.valueBlocks = make([]*trieNode, 0)
index.sparseBlocks = make([]*trieNode, 0)
index.sparseOffset = make([]uint16, 1)
index.lookupBlockIdx = make(map[uint32]int)
index.valueBlockIdx = make(map[uint32]int)
return index
}
func computeOffsets(index *nodeIndex, n *trieNode) int {
if n.leaf {
return n.value
}
hasher := crc32.New(crc32.MakeTable(crc32.IEEE))
// We only index continuation bytes.
for i := 0; i < blockSize; i++ {
v := 0
if nn := n.table[0x80+i]; nn != nil {
v = computeOffsets(index, nn)
}
hasher.Write([]byte{uint8(v >> 8), uint8(v)})
}
h := hasher.Sum32()
if n.isInternal() {
v, ok := index.lookupBlockIdx[h]
if !ok {
v = len(index.lookupBlocks) - blockOffset
index.lookupBlocks = append(index.lookupBlocks, n)
index.lookupBlockIdx[h] = v
}
n.value = v
} else {
v, ok := index.valueBlockIdx[h]
if !ok {
if c := n.countSparseEntries(); c > maxSparseEntries {
v = len(index.valueBlocks) - blockOffset
index.valueBlocks = append(index.valueBlocks, n)
index.valueBlockIdx[h] = v
} else {
v = -len(index.sparseOffset)
index.sparseBlocks = append(index.sparseBlocks, n)
index.sparseOffset = append(index.sparseOffset, uint16(index.sparseCount))
index.sparseCount += c + 1
index.valueBlockIdx[h] = v
}
}
n.value = v
}
return n.value
}
func printValueBlock(nr int, n *trieNode, offset int) {
boff := nr * blockSize
fmt.Printf("\n// Block %#x, offset %#x", nr, boff)
var printnewline bool
for i := 0; i < blockSize; i++ {
if i%6 == 0 {
printnewline = true
}
v := 0
if nn := n.table[i+offset]; nn != nil {
v = nn.value
}
if v != 0 {
if printnewline {
fmt.Printf("\n")
printnewline = false
}
fmt.Printf("%#04x:%#04x, ", boff+i, v)
}
}
}
func printSparseBlock(nr int, n *trieNode) {
boff := -n.value
fmt.Printf("\n// Block %#x, offset %#x", nr, boff)
v := 0
//stride := f(n)
stride := n.mostFrequentStride()
c := n.countSparseEntries()
fmt.Printf("\n{value:%#04x,lo:%#02x},", stride, uint8(c))
for i, nn := range n.table[0x80 : 0x80+blockSize] {
nv := 0
if nn != nil {
nv = nn.value
}
if nv-v != stride {
if v != 0 {
fmt.Printf(",hi:%#02x},", 0x80+i-1)
}
if nv != 0 {
fmt.Printf("\n{value:%#04x,lo:%#02x", nv, nn.b)
}
}
v = nv
}
if v != 0 {
fmt.Printf(",hi:%#02x},", 0x80+blockSize-1)
}
}
func printLookupBlock(nr int, n *trieNode, offset, cutoff int) {
boff := nr * blockSize
fmt.Printf("\n// Block %#x, offset %#x", nr, boff)
var printnewline bool
for i := 0; i < blockSize; i++ {
if i%8 == 0 {
printnewline = true
}
v := 0
if nn := n.table[i+offset]; nn != nil {
v = nn.value
}
if v != 0 {
if v < 0 {
v = -v - 1 + cutoff
}
if printnewline {
fmt.Printf("\n")
printnewline = false
}
fmt.Printf("%#03x:%#02x, ", boff+i, v)
}
}
}
// printTables returns the size in bytes of the generated tables.
func (t *trieNode) printTables(name string) int {
index := newIndex()
// Values for 7-bit ASCII are stored in first two block, followed by nil block.
index.valueBlocks = append(index.valueBlocks, nil, nil, nil)
// First byte of multi-byte UTF-8 codepoints are indexed in 4th block.
index.lookupBlocks = append(index.lookupBlocks, nil, nil, nil, nil)
// Index starter bytes of multi-byte UTF-8.
for i := 0xC0; i < 0x100; i++ {
if t.table[i] != nil {
computeOffsets(index, t.table[i])
}
}
nv := len(index.valueBlocks) * blockSize
fmt.Printf("// %sValues: %d entries, %d bytes\n", name, nv, nv*2)
fmt.Printf("// Block 2 is the null block.\n")
fmt.Printf("var %sValues = [%d]uint16 {", name, nv)
printValueBlock(0, t, 0)
printValueBlock(1, t, 64)
printValueBlock(2, newNode(), 0)
for i := 3; i < len(index.valueBlocks); i++ {
printValueBlock(i, index.valueBlocks[i], 0x80)
}
fmt.Print("\n}\n\n")
ls := len(index.sparseBlocks)
fmt.Printf("// %sSparseOffset: %d entries, %d bytes\n", name, ls, ls*2)
fmt.Printf("var %sSparseOffset = %#v\n\n", name, index.sparseOffset[1:])
ns := index.sparseCount
fmt.Printf("// %sSparseValues: %d entries, %d bytes\n", name, ns, ns*4)
fmt.Printf("var %sSparseValues = [%d]valueRange {", name, ns)
for i, n := range index.sparseBlocks {
printSparseBlock(i, n)
}
fmt.Print("\n}\n\n")
cutoff := len(index.valueBlocks) - blockOffset
ni := len(index.lookupBlocks) * blockSize
fmt.Printf("// %sLookup: %d bytes\n", name, ni)
fmt.Printf("// Block 0 is the null block.\n")
fmt.Printf("var %sLookup = [%d]uint8 {", name, ni)
printLookupBlock(0, newNode(), 0, cutoff)
printLookupBlock(1, newNode(), 0, cutoff)
printLookupBlock(2, newNode(), 0, cutoff)
printLookupBlock(3, t, 0xC0, cutoff)
for i := 4; i < len(index.lookupBlocks); i++ {
printLookupBlock(i, index.lookupBlocks[i], 0x80, cutoff)
}
fmt.Print("\n}\n\n")
fmt.Printf("var %sTrie = trie{ %sLookup[:], %sValues[:], %sSparseValues[:], %sSparseOffset[:], %d}\n\n",
name, name, name, name, name, cutoff)
return nv*2 + ns*4 + ni + ls*2
}
-19
View File
@@ -1,19 +0,0 @@
Copyright (C) 2013 Jakob Borg
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
- The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
-39
View File
@@ -1,39 +0,0 @@
ini [![Build Status](https://drone.io/github.com/calmh/ini/status.png)](https://drone.io/github.com/calmh/ini/latest)
===
Yet another .INI file parser / writer. Created because the existing ones
were either not general enough (allowing easy access to all parts of the
original file) or made annoying assumptions about the format. And
probably equal parts NIH. You might want to just write your own instead
of using this one, you know that's where you'll end up in the end
anyhow.
Documentation
-------------
http://godoc.org/github.com/calmh/ini
Example
-------
```go
fd, _ := os.Open("foo.ini")
cfg := ini.Parse(fd)
fd.Close()
val := cfg.Get("general", "foo")
cfg.Set("general", "bar", "baz")
fd, _ = os.Create("bar.ini")
err := cfg.Write(fd)
if err != nil {
// ...
}
err = fd.Close()
```
License
-------
MIT
-235
View File
@@ -1,235 +0,0 @@
// Package ini provides trivial parsing of .INI format files.
package ini
import (
"bufio"
"fmt"
"io"
"regexp"
"strconv"
"strings"
)
// Config is a parsed INI format file.
type Config struct {
sections []section
comments []string
}
type section struct {
name string
comments []string
options []option
}
type option struct {
name, value string
}
var (
iniSectionRe = regexp.MustCompile(`^\[(.+)\]$`)
iniOptionRe = regexp.MustCompile(`^([^\s=]+)\s*=\s*(.+?)$`)
)
// Sections returns the list of sections in the file.
func (c *Config) Sections() []string {
var sections []string
for _, sect := range c.sections {
sections = append(sections, sect.name)
}
return sections
}
// Options returns the list of options in a given section.
func (c *Config) Options(section string) []string {
var options []string
for _, sect := range c.sections {
if sect.name == section {
for _, opt := range sect.options {
options = append(options, opt.name)
}
break
}
}
return options
}
// OptionMap returns the map option => value for a given section.
func (c *Config) OptionMap(section string) map[string]string {
options := make(map[string]string)
for _, sect := range c.sections {
if sect.name == section {
for _, opt := range sect.options {
options[opt.name] = opt.value
}
break
}
}
return options
}
// Comments returns the list of comments in a given section.
// For the empty string, returns the file comments.
func (c *Config) Comments(section string) []string {
if section == "" {
return c.comments
}
for _, sect := range c.sections {
if sect.name == section {
return sect.comments
}
}
return nil
}
// AddComments appends the comment to the list of comments for the section.
func (c *Config) AddComment(sect, comment string) {
if sect == "" {
c.comments = append(c.comments, comment)
return
}
for i, s := range c.sections {
if s.name == sect {
c.sections[i].comments = append(s.comments, comment)
return
}
}
c.sections = append(c.sections, section{
name: sect,
comments: []string{comment},
})
}
// Parse reads the given io.Reader and returns a parsed Config object.
func Parse(stream io.Reader) Config {
var cfg Config
var curSection string
scanner := bufio.NewScanner(bufio.NewReader(stream))
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if strings.HasPrefix(line, "#") || strings.HasPrefix(line, ";") {
comment := strings.TrimLeft(line, ";# ")
cfg.AddComment(curSection, comment)
} else if len(line) > 0 {
if m := iniSectionRe.FindStringSubmatch(line); len(m) > 0 {
curSection = m[1]
} else if m := iniOptionRe.FindStringSubmatch(line); len(m) > 0 {
key := m[1]
val := m[2]
if !strings.Contains(val, "\"") {
// If val does not contain any quote characers, we can make it
// a quoted string and safely let strconv.Unquote sort out any
// escapes
val = "\"" + val + "\""
}
if val[0] == '"' {
val, _ = strconv.Unquote(val)
}
cfg.Set(curSection, key, val)
}
}
}
return cfg
}
// Write writes the sections and options to the io.Writer in INI format.
func (c *Config) Write(out io.Writer) error {
for _, cmt := range c.comments {
fmt.Fprintln(out, "; "+cmt)
}
if len(c.comments) > 0 {
fmt.Fprintln(out)
}
for _, sect := range c.sections {
fmt.Fprintf(out, "[%s]\n", sect.name)
for _, cmt := range sect.comments {
fmt.Fprintln(out, "; "+cmt)
}
for _, opt := range sect.options {
val := opt.value
if len(val) == 0 {
continue
}
// Quote the string if it begins or ends with space
needsQuoting := val[0] == ' ' || val[len(val)-1] == ' '
if !needsQuoting {
// Quote the string if it contains any unprintable characters
for _, r := range val {
if !strconv.IsPrint(r) {
needsQuoting = true
break
}
}
}
if needsQuoting {
val = strconv.Quote(val)
}
fmt.Fprintf(out, "%s=%s\n", opt.name, val)
}
fmt.Fprintln(out)
}
return nil
}
// Get gets the value from the specified section and key name, or the empty
// string if either the section or the key is missing.
func (c *Config) Get(section, key string) string {
for _, sect := range c.sections {
if sect.name == section {
for _, opt := range sect.options {
if opt.name == key {
return opt.value
}
}
return ""
}
}
return ""
}
// Set sets a value for an option in a section. If the option exists, it's
// value will be overwritten. If the option does not exist, it will be added.
// If the section does not exist, it will be added and the option added to it.
func (c *Config) Set(sectionName, key, value string) {
for i, sect := range c.sections {
if sect.name == sectionName {
for j, opt := range sect.options {
if opt.name == key {
c.sections[i].options[j].value = value
return
}
}
c.sections[i].options = append(sect.options, option{key, value})
return
}
}
c.sections = append(c.sections, section{
name: sectionName,
options: []option{{key, value}},
})
}
// Delete removes the option from the specified section.
func (c *Config) Delete(section, key string) {
for sn, sect := range c.sections {
if sect.name == section {
for i, opt := range sect.options {
if opt.name == key {
c.sections[sn].options = append(sect.options[:i], sect.options[i+1:]...)
return
}
}
return
}
}
}
-214
View File
@@ -1,214 +0,0 @@
package ini_test
import (
"bytes"
"strings"
"testing"
"github.com/calmh/ini"
)
func TestParseValues(t *testing.T) {
strs := []string{
`[general]`,
`k1=v1`,
`k2 = v2`,
` k3 = v3 `,
`k4=" quoted spaces "`,
`k5 = " quoted spaces " `,
`k6 = with\nnewline`,
`k7 = "with\nnewline"`,
`k8 = a "quoted" word`,
`k9 = "a \"quoted\" word"`,
}
buf := bytes.NewBufferString(strings.Join(strs, "\n"))
cfg := ini.Parse(buf)
correct := map[string]string{
"k1": "v1",
"k2": "v2",
"k3": "v3",
"k4": " quoted spaces ",
"k5": " quoted spaces ",
"k6": "with\nnewline",
"k7": "with\nnewline",
"k8": "a \"quoted\" word",
"k9": "a \"quoted\" word",
}
for k, v := range correct {
if v2 := cfg.Get("general", k); v2 != v {
t.Errorf("Incorrect general.%s, %q != %q", k, v2, v)
}
}
if v := cfg.Get("general", "nonexistant"); v != "" {
t.Errorf("Unexpected non-empty value %q", v)
}
}
func TestParseComments(t *testing.T) {
strs := []string{
";file comment 1", // No leading space
"; file comment 2 ", // Trailing space
"; file comment 3", // Multiple leading spaces
"[general]",
"; b general comment 1", // Comments in unsorted order
"somekey = somevalue",
"; a general comment 2",
"[other]",
"; other comment 1", // Comments in section with no values
"; other comment 2",
"[other2]",
"; other2 comment 1",
"; other2 comment 2", // Comments on last section
"somekey = somevalue",
}
buf := bytes.NewBufferString(strings.Join(strs, "\n"))
correct := map[string][]string{
"": []string{"file comment 1", "file comment 2", "file comment 3"},
"general": []string{"b general comment 1", "a general comment 2"},
"other": []string{"other comment 1", "other comment 2"},
"other2": []string{"other2 comment 1", "other2 comment 2"},
}
cfg := ini.Parse(buf)
for section, comments := range correct {
cmts := cfg.Comments(section)
if len(cmts) != len(comments) {
t.Errorf("Incorrect number of comments for section %q: %d != %d", section, len(cmts), len(comments))
} else {
for i := range comments {
if cmts[i] != comments[i] {
t.Errorf("Incorrect comment: %q != %q", cmts[i], comments[i])
}
}
}
}
}
func TestWrite(t *testing.T) {
cfg := ini.Config{}
cfg.Set("general", "k1", "v1")
cfg.Set("general", "k2", "foo bar")
cfg.Set("general", "k3", " foo bar ")
cfg.Set("general", "k4", "foo\nbar")
var out bytes.Buffer
cfg.Write(&out)
correct := `[general]
k1=v1
k2=foo bar
k3=" foo bar "
k4="foo\nbar"
`
if s := out.String(); s != correct {
t.Errorf("Incorrect written .INI:\n%s\ncorrect:\n%s", s, correct)
}
}
func TestSet(t *testing.T) {
buf := bytes.NewBufferString("[general]\nfoo=bar\nfoo2=bar2\n")
cfg := ini.Parse(buf)
cfg.Set("general", "foo", "baz") // Overwrite existing
cfg.Set("general", "baz", "quux") // Create new value
cfg.Set("other", "baz2", "quux2") // Create new section + value
var out bytes.Buffer
cfg.Write(&out)
correct := `[general]
foo=baz
foo2=bar2
baz=quux
[other]
baz2=quux2
`
if s := out.String(); s != correct {
t.Errorf("Incorrect INI after set:\n%s", s)
}
}
func TestDelete(t *testing.T) {
buf := bytes.NewBufferString("[general]\nfoo=bar\nfoo2=bar2\nfoo3=baz\n")
cfg := ini.Parse(buf)
cfg.Delete("general", "foo")
out := new(bytes.Buffer)
cfg.Write(out)
correct := "[general]\nfoo2=bar2\nfoo3=baz\n\n"
if s := out.String(); s != correct {
t.Errorf("Incorrect INI after delete:\n%s", s)
}
buf = bytes.NewBufferString("[general]\nfoo=bar\nfoo2=bar2\nfoo3=baz\n")
cfg = ini.Parse(buf)
cfg.Delete("general", "foo2")
out = new(bytes.Buffer)
cfg.Write(out)
correct = "[general]\nfoo=bar\nfoo3=baz\n\n"
if s := out.String(); s != correct {
t.Errorf("Incorrect INI after delete:\n%s", s)
}
buf = bytes.NewBufferString("[general]\nfoo=bar\nfoo2=bar2\nfoo3=baz\n")
cfg = ini.Parse(buf)
cfg.Delete("general", "foo3")
out = new(bytes.Buffer)
cfg.Write(out)
correct = "[general]\nfoo=bar\nfoo2=bar2\n\n"
if s := out.String(); s != correct {
t.Errorf("Incorrect INI after delete:\n%s", s)
}
}
func TestSetManyEquals(t *testing.T) {
buf := bytes.NewBufferString("[general]\nfoo=bar==\nfoo2=bar2==\n")
cfg := ini.Parse(buf)
cfg.Set("general", "foo", "baz==")
var out bytes.Buffer
cfg.Write(&out)
correct := `[general]
foo=baz==
foo2=bar2==
`
if s := out.String(); s != correct {
t.Errorf("Incorrect INI after set:\n%s", s)
}
}
func TestRewriteDuplicate(t *testing.T) {
buf := bytes.NewBufferString("[general]\nfoo=bar==\nfoo=bar2==\n")
cfg := ini.Parse(buf)
if v := cfg.Get("general", "foo"); v != "bar2==" {
t.Errorf("incorrect get %q", v)
}
var out bytes.Buffer
cfg.Write(&out)
correct := `[general]
foo=bar2==
`
if s := out.String(); s != correct {
t.Errorf("Incorrect INI after set:\n%s", s)
}
}
@@ -1,2 +0,0 @@
inject
inject.test
-20
View File
@@ -1,20 +0,0 @@
The MIT License (MIT)
Copyright (c) 2013 Jeremy Saenz
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-4
View File
@@ -1,4 +0,0 @@
inject
======
Dependency injection for go
-168
View File
@@ -1,168 +0,0 @@
// Package inject provides utilities for mapping and injecting dependencies in various ways.
package inject
import (
"fmt"
"reflect"
)
// Injector represents an interface for mapping and injecting dependencies into structs
// and function arguments.
type Injector interface {
Applicator
Invoker
TypeMapper
// SetParent sets the parent of the injector. If the injector cannot find a
// dependency in its Type map it will check its parent before returning an
// error.
SetParent(Injector)
}
// Applicator represents an interface for mapping dependencies to a struct.
type Applicator interface {
// Maps dependencies in the Type map to each field in the struct
// that is tagged with 'inject'. Returns an error if the injection
// fails.
Apply(interface{}) error
}
// Invoker represents an interface for calling functions via reflection.
type Invoker interface {
// Invoke attempts to call the interface{} provided as a function,
// providing dependencies for function arguments based on Type. Returns
// a slice of reflect.Value representing the returned values of the function.
// Returns an error if the injection fails.
Invoke(interface{}) ([]reflect.Value, error)
}
// TypeMapper represents an interface for mapping interface{} values based on type.
type TypeMapper interface {
// Maps the interface{} value based on its immediate type from reflect.TypeOf.
Map(interface{}) TypeMapper
// Maps the interface{} value based on the pointer of an Interface provided.
// This is really only useful for mapping a value as an interface, as interfaces
// cannot at this time be referenced directly without a pointer.
MapTo(interface{}, interface{}) TypeMapper
// Provides a possibility to directly insert a mapping based on type and value.
// This makes it possible to directly map type arguments not possible to instantiate
// with reflect like unidirectional channels.
Set(reflect.Type, reflect.Value) TypeMapper
// Returns the Value that is mapped to the current type. Returns a zeroed Value if
// the Type has not been mapped.
Get(reflect.Type) reflect.Value
}
type injector struct {
values map[reflect.Type]reflect.Value
parent Injector
}
// InterfaceOf dereferences a pointer to an Interface type.
// It panics if value is not an pointer to an interface.
func InterfaceOf(value interface{}) reflect.Type {
t := reflect.TypeOf(value)
for t.Kind() == reflect.Ptr {
t = t.Elem()
}
if t.Kind() != reflect.Interface {
panic("Called inject.InterfaceOf with a value that is not a pointer to an interface. (*MyInterface)(nil)")
}
return t
}
// New returns a new Injector.
func New() Injector {
return &injector{
values: make(map[reflect.Type]reflect.Value),
}
}
// Invoke attempts to call the interface{} provided as a function,
// providing dependencies for function arguments based on Type.
// Returns a slice of reflect.Value representing the returned values of the function.
// Returns an error if the injection fails.
// It panics if f is not a function
func (inj *injector) Invoke(f interface{}) ([]reflect.Value, error) {
t := reflect.TypeOf(f)
var in = make([]reflect.Value, t.NumIn()) //Panic if t is not kind of Func
for i := 0; i < t.NumIn(); i++ {
argType := t.In(i)
val := inj.Get(argType)
if !val.IsValid() {
return nil, fmt.Errorf("Value not found for type %v", argType)
}
in[i] = val
}
return reflect.ValueOf(f).Call(in), nil
}
// Maps dependencies in the Type map to each field in the struct
// that is tagged with 'inject'.
// Returns an error if the injection fails.
func (inj *injector) Apply(val interface{}) error {
v := reflect.ValueOf(val)
for v.Kind() == reflect.Ptr {
v = v.Elem()
}
if v.Kind() != reflect.Struct {
return nil // Should not panic here ?
}
t := v.Type()
for i := 0; i < v.NumField(); i++ {
f := v.Field(i)
structField := t.Field(i)
if f.CanSet() && structField.Tag == "inject" {
ft := f.Type()
v := inj.Get(ft)
if !v.IsValid() {
return fmt.Errorf("Value not found for type %v", ft)
}
f.Set(v)
}
}
return nil
}
// Maps the concrete value of val to its dynamic type using reflect.TypeOf,
// It returns the TypeMapper registered in.
func (i *injector) Map(val interface{}) TypeMapper {
i.values[reflect.TypeOf(val)] = reflect.ValueOf(val)
return i
}
func (i *injector) MapTo(val interface{}, ifacePtr interface{}) TypeMapper {
i.values[InterfaceOf(ifacePtr)] = reflect.ValueOf(val)
return i
}
// Maps the given reflect.Type to the given reflect.Value and returns
// the Typemapper the mapping has been registered in.
func (i *injector) Set(typ reflect.Type, val reflect.Value) TypeMapper {
i.values[typ] = val
return i
}
func (i *injector) Get(t reflect.Type) reflect.Value {
val := i.values[t]
if !val.IsValid() && i.parent != nil {
val = i.parent.Get(t)
}
return val
}
func (i *injector) SetParent(parent Injector) {
i.parent = parent
}
-142
View File
@@ -1,142 +0,0 @@
package inject_test
import (
"github.com/codegangsta/inject"
"reflect"
"testing"
)
type SpecialString interface {
}
type TestStruct struct {
Dep1 string `inject`
Dep2 SpecialString `inject`
Dep3 string
}
/* Test Helpers */
func expect(t *testing.T, a interface{}, b interface{}) {
if a != b {
t.Errorf("Expected %v (type %v) - Got %v (type %v)", b, reflect.TypeOf(b), a, reflect.TypeOf(a))
}
}
func refute(t *testing.T, a interface{}, b interface{}) {
if a == b {
t.Errorf("Did not expect %v (type %v) - Got %v (type %v)", b, reflect.TypeOf(b), a, reflect.TypeOf(a))
}
}
func Test_InjectorInvoke(t *testing.T) {
injector := inject.New()
expect(t, injector == nil, false)
dep := "some dependency"
injector.Map(dep)
dep2 := "another dep"
injector.MapTo(dep2, (*SpecialString)(nil))
dep3 := make(chan *SpecialString)
dep4 := make(chan *SpecialString)
typRecv := reflect.ChanOf(reflect.RecvDir, reflect.TypeOf(dep3).Elem())
typSend := reflect.ChanOf(reflect.SendDir, reflect.TypeOf(dep4).Elem())
injector.Set(typRecv, reflect.ValueOf(dep3))
injector.Set(typSend, reflect.ValueOf(dep4))
_, err := injector.Invoke(func(d1 string, d2 SpecialString, d3 <-chan *SpecialString, d4 chan<- *SpecialString) {
expect(t, d1, dep)
expect(t, d2, dep2)
expect(t, reflect.TypeOf(d3).Elem(), reflect.TypeOf(dep3).Elem())
expect(t, reflect.TypeOf(d4).Elem(), reflect.TypeOf(dep4).Elem())
expect(t, reflect.TypeOf(d3).ChanDir(), reflect.RecvDir)
expect(t, reflect.TypeOf(d4).ChanDir(), reflect.SendDir)
})
expect(t, err, nil)
}
func Test_InjectorInvokeReturnValues(t *testing.T) {
injector := inject.New()
expect(t, injector == nil, false)
dep := "some dependency"
injector.Map(dep)
dep2 := "another dep"
injector.MapTo(dep2, (*SpecialString)(nil))
result, err := injector.Invoke(func(d1 string, d2 SpecialString) string {
expect(t, d1, dep)
expect(t, d2, dep2)
return "Hello world"
})
expect(t, result[0].String(), "Hello world")
expect(t, err, nil)
}
func Test_InjectorApply(t *testing.T) {
injector := inject.New()
injector.Map("a dep").MapTo("another dep", (*SpecialString)(nil))
s := TestStruct{}
err := injector.Apply(&s)
expect(t, err, nil)
expect(t, s.Dep1, "a dep")
expect(t, s.Dep2, "another dep")
}
func Test_InterfaceOf(t *testing.T) {
iType := inject.InterfaceOf((*SpecialString)(nil))
expect(t, iType.Kind(), reflect.Interface)
iType = inject.InterfaceOf((**SpecialString)(nil))
expect(t, iType.Kind(), reflect.Interface)
// Expecting nil
defer func() {
rec := recover()
refute(t, rec, nil)
}()
iType = inject.InterfaceOf((*testing.T)(nil))
}
func Test_InjectorSet(t *testing.T) {
injector := inject.New()
typ := reflect.TypeOf("string")
typSend := reflect.ChanOf(reflect.SendDir, typ)
typRecv := reflect.ChanOf(reflect.RecvDir, typ)
// instantiating unidirectional channels is not possible using reflect
// http://golang.org/src/pkg/reflect/value.go?s=60463:60504#L2064
chanRecv := reflect.MakeChan(reflect.ChanOf(reflect.BothDir, typ), 0)
chanSend := reflect.MakeChan(reflect.ChanOf(reflect.BothDir, typ), 0)
injector.Set(typSend, chanSend)
injector.Set(typRecv, chanRecv)
expect(t, injector.Get(typSend).IsValid(), true)
expect(t, injector.Get(typRecv).IsValid(), true)
expect(t, injector.Get(chanSend.Type()).IsValid(), false)
}
func Test_InjectorGet(t *testing.T) {
injector := inject.New()
injector.Map("some dependency")
expect(t, injector.Get(reflect.TypeOf("string")).IsValid(), true)
expect(t, injector.Get(reflect.TypeOf(11)).IsValid(), false)
}
func Test_InjectorSetParent(t *testing.T) {
injector := inject.New()
injector.MapTo("another dep", (*SpecialString)(nil))
injector2 := inject.New()
injector2.SetParent(injector)
expect(t, injector2.Get(inject.InterfaceOf((*SpecialString)(nil))).IsValid(), true)
}
@@ -1,23 +0,0 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
*.test
-20
View File
@@ -1,20 +0,0 @@
The MIT License (MIT)
Copyright (c) 2013 Jeremy Saenz
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
-345
View File
@@ -1,345 +0,0 @@
# Martini [![wercker status](https://app.wercker.com/status/174bef7e3c999e103cacfe2770102266 "wercker status")](https://app.wercker.com/project/bykey/174bef7e3c999e103cacfe2770102266) [![GoDoc](https://godoc.org/github.com/codegangsta/martini?status.png)](http://godoc.org/github.com/codegangsta/martini)
Martini is a powerful package for quickly writing modular web applications/services in Golang.
Language Translations: [Simplified Chinese (zh_CN)](translations/README_zh_cn.md)
## Getting Started
After installing Go and setting up your [GOPATH](http://golang.org/doc/code.html#GOPATH), create your first `.go` file. We'll call it `server.go`.
~~~ go
package main
import "github.com/codegangsta/martini"
func main() {
m := martini.Classic()
m.Get("/", func() string {
return "Hello world!"
})
m.Run()
}
~~~
Then install the Martini package (**go 1.1** and greater is required):
~~~
go get github.com/codegangsta/martini
~~~
Then run your server:
~~~
go run server.go
~~~
You will now have a Martini webserver running on `localhost:3000`.
## Getting Help
Join the [Mailing list](https://groups.google.com/forum/#!forum/martini-go)
Watch the [Demo Video](http://martini.codegangsta.io/#demo)
## Features
* Extremely simple to use.
* Non-intrusive design.
* Plays nice with other Golang packages.
* Awesome path matching and routing.
* Modular design - Easy to add functionality, easy to rip stuff out.
* Lots of good handlers/middlewares to use.
* Great 'out of the box' feature set.
* **Fully compatible with the [http.HandlerFunc](http://godoc.org/net/http#HandlerFunc) interface.**
## More Middleware
For more middleware and functionality, check out the repositories in the [martini-contrib](https://github.com/martini-contrib) organization.
## Table of Contents
* [Classic Martini](#classic-martini)
* [Handlers](#handlers)
* [Routing](#routing)
* [Services](#services)
* [Serving Static Files](#serving-static-files)
* [Middleware Handlers](#middleware-handlers)
* [Next()](#next)
* [Martini Env](#martini-env)
* [FAQ](#faq)
## Classic Martini
To get up and running quickly, [martini.Classic()](http://godoc.org/github.com/codegangsta/martini#Classic) provides some reasonable defaults that work well for most web applications:
~~~ go
m := martini.Classic()
// ... middleware and routing goes here
m.Run()
~~~
Below is some of the functionality [martini.Classic()](http://godoc.org/github.com/codegangsta/martini#Classic) pulls in automatically:
* Request/Response Logging - [martini.Logger](http://godoc.org/github.com/codegangsta/martini#Logger)
* Panic Recovery - [martini.Recovery](http://godoc.org/github.com/codegangsta/martini#Recovery)
* Static File serving - [martini.Static](http://godoc.org/github.com/codegangsta/martini#Static)
* Routing - [martini.Router](http://godoc.org/github.com/codegangsta/martini#Router)
### Handlers
Handlers are the heart and soul of Martini. A handler is basically any kind of callable function:
~~~ go
m.Get("/", func() {
println("hello world")
})
~~~
#### Return Values
If a handler returns something, Martini will write the result to the current [http.ResponseWriter](http://godoc.org/net/http#ResponseWriter) as a string:
~~~ go
m.Get("/", func() string {
return "hello world" // HTTP 200 : "hello world"
})
~~~
You can also optionally return a status code:
~~~ go
m.Get("/", func() (int, string) {
return 418, "i'm a teapot" // HTTP 418 : "i'm a teapot"
})
~~~
#### Service Injection
Handlers are invoked via reflection. Martini makes use of *Dependency Injection* to resolve dependencies in a Handlers argument list. **This makes Martini completely compatible with golang's `http.HandlerFunc` interface.**
If you add an argument to your Handler, Martini will search its list of services and attempt to resolve the dependency via type assertion:
~~~ go
m.Get("/", func(res http.ResponseWriter, req *http.Request) { // res and req are injected by Martini
res.WriteHeader(200) // HTTP 200
})
~~~
The following services are included with [martini.Classic()](http://godoc.org/github.com/codegangsta/martini#Classic):
* [*log.Logger](http://godoc.org/log#Logger) - Global logger for Martini.
* [martini.Context](http://godoc.org/github.com/codegangsta/martini#Context) - http request context.
* [martini.Params](http://godoc.org/github.com/codegangsta/martini#Params) - `map[string]string` of named params found by route matching.
* [martini.Routes](http://godoc.org/github.com/codegangsta/martini#Routes) - Route helper service.
* [http.ResponseWriter](http://godoc.org/net/http/#ResponseWriter) - http Response writer interface.
* [*http.Request](http://godoc.org/net/http/#Request) - http Request.
### Routing
In Martini, a route is an HTTP method paired with a URL-matching pattern.
Each route can take one or more handler methods:
~~~ go
m.Get("/", func() {
// show something
})
m.Patch("/", func() {
// update something
})
m.Post("/", func() {
// create something
})
m.Put("/", func() {
// replace something
})
m.Delete("/", func() {
// destroy something
})
m.Options("/", func() {
// http options
})
m.NotFound(func() {
// handle 404
})
~~~
Routes are matched in the order they are defined. The first route that
matches the request is invoked.
Route patterns may include named parameters, accessible via the [martini.Params](http://godoc.org/github.com/codegangsta/martini#Params) service:
~~~ go
m.Get("/hello/:name", func(params martini.Params) string {
return "Hello " + params["name"]
})
~~~
Routes can be matched with regular expressions and globs as well:
~~~ go
m.Get("/hello/**", func(params martini.Params) string {
return "Hello " + params["_1"]
})
~~~
Route handlers can be stacked on top of each other, which is useful for things like authentication and authorization:
~~~ go
m.Get("/secret", authorize, func() {
// this will execute as long as authorize doesn't write a response
})
~~~
Route groups can be added too using the Group method.
~~~ go
m.Group("/books", func(r Router) {
r.Get("/:id", GetBooks)
r.Post("/new", NewBook)
r.Put("/update/:id", UpdateBook)
r.Delete("/delete/:id", DeleteBook)
})
~~~
Just like you can pass middlewares to a handler you can pass middlewares to groups.
~~~ go
m.Group("/books", func(r Router) {
r.Get("/:id", GetBooks)
r.Post("/new", NewBook)
r.Put("/update/:id", UpdateBook)
r.Delete("/delete/:id", DeleteBook)
}, MyMiddleware1, MyMiddleware2)
~~~
### Services
Services are objects that are available to be injected into a Handler's argument list. You can map a service on a *Global* or *Request* level.
#### Global Mapping
A Martini instance implements the inject.Injector interface, so mapping a service is easy:
~~~ go
db := &MyDatabase{}
m := martini.Classic()
m.Map(db) // the service will be available to all handlers as *MyDatabase
// ...
m.Run()
~~~
#### Request-Level Mapping
Mapping on the request level can be done in a handler via [martini.Context](http://godoc.org/github.com/codegangsta/martini#Context):
~~~ go
func MyCustomLoggerHandler(c martini.Context, req *http.Request) {
logger := &MyCustomLogger{req}
c.Map(logger) // mapped as *MyCustomLogger
}
~~~
#### Mapping values to Interfaces
One of the most powerful parts about services is the ability to map a service to an interface. For instance, if you wanted to override the [http.ResponseWriter](http://godoc.org/net/http#ResponseWriter) with an object that wrapped it and performed extra operations, you can write the following handler:
~~~ go
func WrapResponseWriter(res http.ResponseWriter, c martini.Context) {
rw := NewSpecialResponseWriter(res)
c.MapTo(rw, (*http.ResponseWriter)(nil)) // override ResponseWriter with our wrapper ResponseWriter
}
~~~
### Serving Static Files
A [martini.Classic()](http://godoc.org/github.com/codegangsta/martini#Classic) instance automatically serves static files from the "public" directory in the root of your server.
You can serve from more directories by adding more [martini.Static](http://godoc.org/github.com/codegangsta/martini#Static) handlers.
~~~ go
m.Use(martini.Static("assets")) // serve from the "assets" directory as well
~~~
## Middleware Handlers
Middleware Handlers sit between the incoming http request and the router. In essence they are no different than any other Handler in Martini. You can add a middleware handler to the stack like so:
~~~ go
m.Use(func() {
// do some middleware stuff
})
~~~
You can have full control over the middleware stack with the `Handlers` function. This will replace any handlers that have been previously set:
~~~ go
m.Handlers(
Middleware1,
Middleware2,
Middleware3,
)
~~~
Middleware Handlers work really well for things like logging, authorization, authentication, sessions, gzipping, error pages and any other operations that must happen before or after an http request:
~~~ go
// validate an api key
m.Use(func(res http.ResponseWriter, req *http.Request) {
if req.Header.Get("X-API-KEY") != "secret123" {
res.WriteHeader(http.StatusUnauthorized)
}
})
~~~
### Next()
[Context.Next()](http://godoc.org/github.com/codegangsta/martini#Context) is an optional function that Middleware Handlers can call to yield the until after the other Handlers have been executed. This works really well for any operations that must happen after an http request:
~~~ go
// log before and after a request
m.Use(func(c martini.Context, log *log.Logger){
log.Println("before a request")
c.Next()
log.Println("after a request")
})
~~~
## Martini Env
Some Martini handlers make use of the `martini.Env` global variable to provide special functionality for development environments vs production environments. It is reccomended that the `MARTINI_ENV=production` environment variable to be set when deploying a Martini server into a production environment.
## FAQ
### Where do I find middleware X?
Start by looking in the [martini-contrib](https://github.com/martini-contrib) projects. If it is not there feel free to contact a martini-contrib team member about adding a new repo to the organization.
* [auth](https://github.com/martini-contrib/auth) - Handlers for authentication.
* [binding](https://github.com/martini-contrib/binding) - Handler for mapping/validating a raw request into a structure.
* [gzip](https://github.com/martini-contrib/gzip) - Handler for adding gzip compress to requests
* [render](https://github.com/martini-contrib/render) - Handler that provides a service for easily rendering JSON and HTML templates.
* [acceptlang](https://github.com/martini-contrib/acceptlang) - Handler for parsing the `Accept-Language` HTTP header.
* [sessions](https://github.com/martini-contrib/sessions) - Handler that provides a Session service.
* [strip](https://github.com/martini-contrib/strip) - URL Prefix stripping.
* [method](https://github.com/martini-contrib/method) - HTTP method overriding via Header or form fields.
* [secure](https://github.com/martini-contrib/secure) - Implements a few quick security wins.
* [encoder](https://github.com/martini-contrib/encoder) - Encoder service for rendering data in several formats and content negotiation.
* [cors](https://github.com/martini-contrib/cors) - Handler that enables CORS support.
* [oauth2](https://github.com/martini-contrib/oauth2) - Handler that provides OAuth 2.0 login for Martini apps. Google Sign-in, Facebook Connect and Github login is supported.
### How do I integrate with existing servers?
A Martini instance implements `http.Handler`, so it can easily be used to serve subtrees
on existing Go servers. For example this is a working Martini app for Google App Engine:
~~~ go
package hello
import (
"net/http"
"github.com/codegangsta/martini"
)
func init() {
m := martini.Classic()
m.Get("/", func() string {
return "Hello world!"
})
http.Handle("/", m)
}
~~~
### How do I change the port/host?
Martini's `Run` function looks for the PORT and HOST environment variables and uses those. Otherwise Martini will default to localhost:3000.
To have more flexibility over port and host, use the `http.ListenAndServe` function instead.
~~~ go
m := martini.Classic()
// ...
log.Fatal(http.ListenAndServe(":8080", m))
~~~
### Live code reload?
[gin](https://github.com/codegangsta/gin) and [fresh](https://github.com/pilu/fresh) both live reload martini apps.
## Contributing
Martini is meant to be kept tiny and clean. Most contributions should end up in a repository in the [martini-contrib](https://github.com/martini-contrib) organization. If you do have a contribution for the core of Martini feel free to put up a Pull Request.
## About
Inspired by [express](https://github.com/visionmedia/express) and [sinatra](https://github.com/sinatra/sinatra)
Martini is obsessively designed by none other than the [Code Gangsta](http://codegangsta.io/)

Some files were not shown because too many files have changed in this diff Show More