本站源代码
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

99 lines
3.9KB

  1. // Copyright 2011 The Snappy-Go Authors. All rights reserved.
  2. // Use of this source code is governed by a BSD-style
  3. // license that can be found in the LICENSE file.
  4. // Package snappy implements the Snappy compression format. It aims for very
  5. // high speeds and reasonable compression.
  6. //
  7. // There are actually two Snappy formats: block and stream. They are related,
  8. // but different: trying to decompress block-compressed data as a Snappy stream
  9. // will fail, and vice versa. The block format is the Decode and Encode
  10. // functions and the stream format is the Reader and Writer types.
  11. //
  12. // The block format, the more common case, is used when the complete size (the
  13. // number of bytes) of the original data is known upfront, at the time
  14. // compression starts. The stream format, also known as the framing format, is
  15. // for when that isn't always true.
  16. //
  17. // The canonical, C++ implementation is at https://github.com/google/snappy and
  18. // it only implements the block format.
  19. package snappy // import "github.com/golang/snappy"
  20. import (
  21. "hash/crc32"
  22. )
  23. /*
  24. Each encoded block begins with the varint-encoded length of the decoded data,
  25. followed by a sequence of chunks. Chunks begin and end on byte boundaries. The
  26. first byte of each chunk is broken into its 2 least and 6 most significant bits
  27. called l and m: l ranges in [0, 4) and m ranges in [0, 64). l is the chunk tag.
  28. Zero means a literal tag. All other values mean a copy tag.
  29. For literal tags:
  30. - If m < 60, the next 1 + m bytes are literal bytes.
  31. - Otherwise, let n be the little-endian unsigned integer denoted by the next
  32. m - 59 bytes. The next 1 + n bytes after that are literal bytes.
  33. For copy tags, length bytes are copied from offset bytes ago, in the style of
  34. Lempel-Ziv compression algorithms. In particular:
  35. - For l == 1, the offset ranges in [0, 1<<11) and the length in [4, 12).
  36. The length is 4 + the low 3 bits of m. The high 3 bits of m form bits 8-10
  37. of the offset. The next byte is bits 0-7 of the offset.
  38. - For l == 2, the offset ranges in [0, 1<<16) and the length in [1, 65).
  39. The length is 1 + m. The offset is the little-endian unsigned integer
  40. denoted by the next 2 bytes.
  41. - For l == 3, this tag is a legacy format that is no longer issued by most
  42. encoders. Nonetheless, the offset ranges in [0, 1<<32) and the length in
  43. [1, 65). The length is 1 + m. The offset is the little-endian unsigned
  44. integer denoted by the next 4 bytes.
  45. */
  46. const (
  47. tagLiteral = 0x00
  48. tagCopy1 = 0x01
  49. tagCopy2 = 0x02
  50. tagCopy4 = 0x03
  51. )
  52. const (
  53. checksumSize = 4
  54. chunkHeaderSize = 4
  55. magicChunk = "\xff\x06\x00\x00" + magicBody
  56. magicBody = "sNaPpY"
  57. // maxBlockSize is the maximum size of the input to encodeBlock. It is not
  58. // part of the wire format per se, but some parts of the encoder assume
  59. // that an offset fits into a uint16.
  60. //
  61. // Also, for the framing format (Writer type instead of Encode function),
  62. // https://github.com/google/snappy/blob/master/framing_format.txt says
  63. // that "the uncompressed data in a chunk must be no longer than 65536
  64. // bytes".
  65. maxBlockSize = 65536
  66. // maxEncodedLenOfMaxBlockSize equals MaxEncodedLen(maxBlockSize), but is
  67. // hard coded to be a const instead of a variable, so that obufLen can also
  68. // be a const. Their equivalence is confirmed by
  69. // TestMaxEncodedLenOfMaxBlockSize.
  70. maxEncodedLenOfMaxBlockSize = 76490
  71. obufHeaderLen = len(magicChunk) + checksumSize + chunkHeaderSize
  72. obufLen = obufHeaderLen + maxEncodedLenOfMaxBlockSize
  73. )
  74. const (
  75. chunkTypeCompressedData = 0x00
  76. chunkTypeUncompressedData = 0x01
  77. chunkTypePadding = 0xfe
  78. chunkTypeStreamIdentifier = 0xff
  79. )
  80. var crcTable = crc32.MakeTable(crc32.Castagnoli)
  81. // crc implements the checksum specified in section 3 of
  82. // https://github.com/google/snappy/blob/master/framing_format.txt
  83. func crc(b []byte) uint32 {
  84. c := crc32.Update(0, crcTable, b)
  85. return uint32(c>>15|c<<17) + 0xa282ead8
  86. }
上海开阖软件有限公司 沪ICP备12045867号-1