usbcore.utils package

Submodules

usbcore.utils.CrcMoose3 module

This module can model common CRC algorithms given the set of defining parameters. This is intended to be easy to use for experimentation rather than optimized for speed. It is slow even for a native Python CRC implementation.

Several common CRC algorithms are predefined in this module.

authors:Ray Burr
license:MIT License
contact:http://www.nightmare.com/~ryb/

Examples

>>> '%X' % CRC32.calcString('123456789')
'CBF43926'

This test function runs all of the defined algorithms on the test input string ‘123456789’:

>>> _printResults()
CRC-5-USB: 19
CRC-8-SMBUS: F4
CRC-15: 059E
CRC-16: BB3D
CRC-16-USB: B4C8
CRC-CCITT: 29B1
CRC-HDLC: 906E
CRC-24: 21CF02
CRC-32: CBF43926
CRC-32C: E3069283
CRC-64: 46A5A9388A5BEFFE
CRC-256: 79B96BDC0C519B239BE759EC0688C86FD25A3F4DF1E7F054AD1F923D0739DAC8

Calculating in parts:

>>> value = CRC32.calcString('1234')
>>> '%X' % CRC32.calcString('56789', value)
'CBF43926'

Or, done a different way:

>>> crc = CrcRegister(CRC32)
>>> crc.takeString('1234')
>>> crc.takeString('56789')
>>> '%X' % crc.getFinalValue()
'CBF43926'

Inversion of a CRC function:

>>> CRC_CCITT.reverse().reflect().calcWord(54321, 16, 0)
1648
>>> CRC_CCITT.calcWord(_, 16, 0)
54321

A 15-bit CRC is used in CAN protocols. The following sample CAN frame (in binary here) is converted to hexadecimal for the calcWord call. The bits after the 15-bit CRC are not included in the CRC:

0 11101000001 0 0 0 0001 00010010 011000010111011 1 1 1 1111111

This sample CAN frame was found in this paper: <http://www.anthony-marino.com/documents/HDL_implementation_CAN.pdf>

>>> '%X' % CRC15.calcWord(0x3A08112, 27)
'30BB'

If the CRC is included, the remainder should always be zero:

>>> print(CRC15.calcWord(0x1D0408930BB, 42))
0

A 5-bit CRC is used some kinds of USB packets. Here is a sample start-of-frame packet:

10100101 01100111000 01111

(found at <http://www.nital.com/corporate/usb2snooper.html>)

The first field is the PID (not included in the CRC), the next 11-bit field is the frame number (0xE6, LSb-first order), and the final five bits are the CRC (0x1E, LSb-first order).

>>> '%X' % CRC5_USB.calcWord(0xE6, 11)
'1E'
class usbcore.utils.CrcMoose3.CrcAlgorithm(width, polynomial, name=None, seed=0, lsbFirst=False, lsbFirstData=None, xorMask=0)[source]

Bases: object

Represents the parameters of a CRC algorithm.

calcString(s, value=None)[source]

Calculate the CRC of the 8-bit string s.

calcWord(word, width, value=None)[source]

Calculate the CRC of the integer word as a sequence of width bits.

reflect()[source]

Return the algorithm with the bit-order reversed.

reverse()[source]

Return the algorithm with the reverse polynomial.

class usbcore.utils.CrcMoose3.CrcRegister(crcAlgorithm, value=None)[source]

Bases: object

Holds the intermediate state of the CRC algorithm.

reset()[source]

Reset the state of the register with the default seed value.

takeBit(bit)[source]

Process a single input bit.

takeWord(word, width=8)[source]

Process a binary input word.

Parameters:
  • word (an integer) – The input word. Since this can be a Python long, there is no coded limit to the number of bits the word can represent.
  • width (an integer) – The number of bits word represents.
takeString(s)[source]

Process a string as input. It is handled as a sequence of 8-bit integers.

getValue()[source]

Return the current value of the register as an integer.

getFinalValue()[source]

Return the current value of the register as an integer with xorMask applied. This can be used after all input data is processed to obtain the final result.

usbcore.utils.CrcMoose3.reflect(value, width)[source]
usbcore.utils.CrcMoose3.formatBinaryString(value, width)[source]
usbcore.utils.CrcMoose3.CRC32 = <usbcore.utils.CrcMoose3.CrcAlgorithm "CRC-32" @ 0x7fc5ab324b70>

Same CRC algorithm as Python’s zlib.crc32

usbcore.utils.CrcMoose3.CRC16_USB = <usbcore.utils.CrcMoose3.CrcAlgorithm "CRC-16-USB" @ 0x7fc5aaf77be0>

Used in USB data packets.

usbcore.utils.CrcMoose3.CRC_HDLC = <usbcore.utils.CrcMoose3.CrcAlgorithm "CRC-HDLC" @ 0x7fc5aaf77c50>

This is the algorithm used in X.25 and for the HDLC 2-byte FCS.

usbcore.utils.CrcMoose3.CRC8_SMBUS = <usbcore.utils.CrcMoose3.CrcAlgorithm "CRC-8-SMBUS" @ 0x7fc5aaf77c88>

Used in ATM HEC and SMBus.

usbcore.utils.CrcMoose3.CRC24 = <usbcore.utils.CrcMoose3.CrcAlgorithm "CRC-24" @ 0x7fc5aaf77cc0>

Used in RFC-2440 and MIL STD 188-184.

usbcore.utils.CrcMoose3.CRC15 = <usbcore.utils.CrcMoose3.CrcAlgorithm "CRC-15" @ 0x7fc5aaf77cf8>

Used in Controller Area Network frames.

usbcore.utils.CrcMoose3.CRC32C = <usbcore.utils.CrcMoose3.CrcAlgorithm "CRC-32C" @ 0x7fc5aaf77d30>

Used in iSCSI (RFC-3385); usually credited to Guy Castagnoli.

usbcore.utils.CrcMoose3.CRC5_USB = <usbcore.utils.CrcMoose3.CrcAlgorithm "CRC-5-USB" @ 0x7fc5aaf77d68>

CRC used in USB Token and Start-Of-Frame packets

usbcore.utils.CrcMoose3.CRC64 = <usbcore.utils.CrcMoose3.CrcAlgorithm "CRC-64" @ 0x7fc5aaf77da0>

ISO 3309

usbcore.utils.asserts module

usbcore.utils.asserts.assertMultiLineEqualSideBySide(expected, actual, msg)[source]

usbcore.utils.bits module

usbcore.utils.bits.int_to_bits(i, width=None)[source]

Convert an int to list of bits (LSB first).

l[0] == LSB l[-1] == MSB

0b10000
|
— LSB
—— MSB
>>> int_to_bits(0b1)
[1]
>>> int_to_bits(0b0)
[0]
>>> int_to_bits(0b100)
[0, 0, 1]
>>> int_to_bits(0b100, 4)
[0, 0, 1, 0]
>>> int_to_bits(0b100, 8)
[0, 0, 1, 0, 0, 0, 0, 0]
usbcore.utils.bits.bits_to_int(bits)[source]

Convert a list of bits (LSB first) to an int.

l[0] == LSB l[-1] == MSB

0b10000
|
— LSB
—— MSB
>>> bits_to_int([1])
1
>>> bits_to_int([0])
0
>>> bin(bits_to_int([0, 0, 1]))
'0b100'
>>> bin(bits_to_int([0, 0, 1, 0]))
'0b100'
>>> bin(bits_to_int([0, 1, 0, 1]))
'0b1010'
>>> bin(bits_to_int([0, 1, 0, 1, 0, 0, 0]))
'0b1010'
>>> bin(bits_to_int([0, 0, 0, 0, 0, 1, 0, 1]))
'0b10100000'
usbcore.utils.bits.int_to_rbits(i, width=None)[source]

Convert an int to list of bits (MSB first).

l[0] == MSB l[-1] == LSB

0b10000
|
— LSB
—— MSB
>>> int_to_rbits(0b1)
[1]
>>> int_to_rbits(0b0)
[0]
>>> int_to_rbits(0b100)
[1, 0, 0]
>>> int_to_rbits(0b100, 4)
[0, 1, 0, 0]
>>> int_to_rbits(0b100, 8)
[0, 0, 0, 0, 0, 1, 0, 0]
usbcore.utils.bits.rbits_to_int(rbits)[source]

Convert a list of bits (MSB first) to an int.

l[0] == MSB l[-1] == LSB

0b10000
|
— LSB
—— MSB
>>> rbits_to_int([1])
1
>>> rbits_to_int([0])
0
>>> bin(rbits_to_int([1, 0, 0]))
'0b100'
>>> bin(rbits_to_int([1, 0, 1, 0]))
'0b1010'
>>> bin(rbits_to_int([1, 0, 1, 0, 0, 0, 0, 0]))
'0b10100000'
usbcore.utils.bits.get_bit(epaddr, v)[source]
>>> get_bit(0, 0b11)
True
>>> get_bit(0, 0b10)
False
>>> get_bit(0, 0b101)
True
>>> get_bit(1, 0b101)
False
usbcore.utils.bits.set_bit(current, epaddr, v)[source]
>>> bin(set_bit(0, 0, 1))
'0b1'
>>> bin(set_bit(0, 2, 1))
'0b100'
>>> bin(set_bit(0b1000, 2, 1))
'0b1100'
>>> bin(set_bit(0b1100, 2, 0))
'0b1000'
>>> bin(set_bit(0b1101, 2, 0))
'0b1001'

usbcore.utils.packet module

usbcore.utils.packet.b(s)[source]

Byte string with LSB first into an integer.

>>> b("1")
1
>>> b("01")
2
>>> b("101")
5
usbcore.utils.packet.encode_data(data)[source]

Converts array of 8-bit ints into string of 0s and 1s.

usbcore.utils.packet.encode_pid(value)[source]
usbcore.utils.packet.crc5(nibbles)[source]
>>> hex(crc5([0, 0]))
'0x1'
>>> hex(crc5([3, 0]))
'0x13'
usbcore.utils.packet.crc5_token(addr, ep)[source]
>>> hex(crc5_token(0, 0))
'0x2'
>>> hex(crc5_token(92, 0))
'0x1c'
>>> hex(crc5_token(3, 0))
'0xa'
>>> hex(crc5_token(56, 4))
'0xb'
usbcore.utils.packet.crc5_sof(v)[source]
>>> hex(crc5_sof(1429))
'0x1'
>>> hex(crc5_sof(1013))
'0x5'
usbcore.utils.packet.crc16(input_data)[source]
usbcore.utils.packet.nrzi(data, cycles=4, init='J')[source]

Converts string of 0s and 1s into NRZI encoded string.

>>> nrzi("11 00000001", 1)
'JJ KJKJKJKK'

It will do bit stuffing. >>> nrzi(“1111111111”, 1) ‘JJJJJJKKKKK’

Support single ended zero >>> nrzi(“1111111__”, 1) ‘JJJJJJKK__’

Support pre-encoded mixing. >>> nrzi(“11kkj11__”, 1) ‘JJKKJJJ__’

Supports wider clock widths >>> nrzi(“101”, 4) ‘JJJJKKKKKKKK’

usbcore.utils.packet.sync()[source]
usbcore.utils.packet.eop()[source]
usbcore.utils.packet.wrap_packet(data, cycles=4)[source]

Add the sync + eop sections and do nrzi encoding.

>>> wrap_packet(handshake_packet(PID.ACK), cycles=1)
'KJKJKJKKJJKJJKKK__J'
>>> wrap_packet(token_packet(PID.SETUP, 0, 0), cycles=1)
'KJKJKJKKKJJJKKJKJKJKJKJKJKJKKJKJ__J'
>>> wrap_packet(data_packet(PID.DATA0, [5, 6]), cycles=1)
'KJKJKJKKKKJKJKKKKJJKJKJKJJJKJKJKKJJJJJJKKJJJJKJK__J'
>>> wrap_packet(data_packet(PID.DATA0, [0x1]), cycles=1)
'KJKJKJKKKKJKJKKKKJKJKJKJJKJKJKJJJJJJJKKKJ__J'
usbcore.utils.packet.token_packet(pid, addr, endp)[source]

Create a token packet for testing.

sync, pid, addr (7bit), endp(4bit), crc5(5bit), eop

>>> token_packet(PID.SETUP, 0x0, 0x0)
'101101000000000000001000'
PPPPPPPP - 8 bits - PID
AAAAAAA - 7 bits - ADDR
EEEE - 4 bits - EP
CCCCC - 5 bits - CRC
>>> token_packet(PID.IN, 0x3, 0x0) # 0x0A
'100101101100000000001010'

>>> token_packet(PID.OUT, 0x3a, 0xa)
'100001110101110010111100'

>>> token_packet(PID.SETUP, 0x70, 0xa)
'101101000000111010110101'

>>> token_packet(PID.SETUP, 40, 2)
'101101000001010010000011'

>>> token_packet(PID.SETUP, 28, 2)
'101101000011100010001001'
PPPPPPPP - 8 bits - PID
AAAAAAA - 7 bits - ADDR
EEEE - 4 bits - EP
CCCCC - 5 bits - CRC
usbcore.utils.packet.data_packet(pid, payload)[source]

Create a data packet for testing.

sync, pid, data, crc16, eop FIXME: data should be multiples of 8?

>>> data_packet(PID.DATA0, [0x80, 0x06, 0x03, 0x03, 0x09, 0x04, 0x00, 0x02])
'1100001100000001011000001100000011000000100100000010000000000000010000000110101011011100'

>>> data_packet(PID.DATA1, [])
'110100100000000000000000'
usbcore.utils.packet.handshake_packet(pid)[source]

Create a handshake packet for testing.

sync, pid, eop ack / nak / stall / nyet (high speed only)

>>> handshake_packet(PID.ACK)
'01001011'
>>> handshake_packet(PID.NAK)
'01011010'
usbcore.utils.packet.sof_packet(frame)[source]

Create a SOF packet for testing.

sync, pid, frame no (11bits), crc5(5bits), eop

>>> sof_packet(1)
'101001011000000000010111'

>>> sof_packet(100)
'101001010010011000011111'

>>> sof_packet(257)
'101001011000000010000011'

>>> sof_packet(1429)
'101001011010100110110000'

>>> sof_packet(2**11 - 2)
'101001010111111111111101'
usbcore.utils.packet.diff(value)[source]

Convert J/K encoding into bits for P/N diff pair.

>>> diff('KJ_')
('010', '100')

>>> # Convert ACK handshake packet
>>> p,n = diff('KJKJKJKKJJKJJKKK__J')
>>> p
'0101010011011000001'
>>> n
'1010101100100111000'
usbcore.utils.packet.undiff(usbp, usbn)[source]

Convert P/N diff pair bits into J/K encoding.

>>> from usbcore.utils.pprint import pp_packet
>>> undiff(
...   #EJK_
...   '1100', # p
...   '1010', # n
... )
'EJK_'
>>> print(pp_packet(undiff(
...   #KJKJKJKKJJKJJKKK__J - ACK handshake packet
...   '0101010011011000001', # p
...   '1010101100100111000', # n
... ), cycles=1))
-
K 1 Sync
J 2 Sync
K 3 Sync
J 4 Sync
K 5 Sync
J 6 Sync
K 7 Sync
K 8 Sync
-
J 1 PID (PID.ACK)
J 2 PID
K 3 PID
J 4 PID
J 5 PID
K 6 PID
K 7 PID
K 8 PID
-
_ SE0
_ SE0
J END
>>> print(pp_packet(undiff(*diff(wrap_packet(sof_packet(0))))))
----
KKKK 1 Sync
JJJJ 2 Sync
KKKK 3 Sync
JJJJ 4 Sync
KKKK 5 Sync
JJJJ 6 Sync
KKKK 7 Sync
KKKK 8 Sync
----
KKKK 1 PID (PID.SOF)
JJJJ 2 PID
JJJJ 3 PID
KKKK 4 PID
JJJJ 5 PID
JJJJ 6 PID
KKKK 7 PID
KKKK 8 PID
----
JJJJ  1 Frame #
KKKK  2 Frame #
JJJJ  3 Frame #
KKKK  4 Frame #
JJJJ  5 Frame #
KKKK  6 Frame #
JJJJ  7 Frame #
KKKK  8 Frame #
----
JJJJ  9 Frame #
JJJJ 10 Frame #
KKKK 11 Frame #
JJJJ 1 CRC5
KKKK 2 CRC5
JJJJ 3 CRC5
KKKK 4 CRC5
JJJJ 5 CRC5
----
____ SE0
____ SE0
JJJJ END

usbcore.utils.pprint module

usbcore.utils.pprint.pp_packet(p, cycles=4)[source]
>>> print(pp_packet(wrap_packet(handshake_packet(PID.ACK), cycles=1), cycles=1))
-
K 1 Sync
J 2 Sync
K 3 Sync
J 4 Sync
K 5 Sync
J 6 Sync
K 7 Sync
K 8 Sync
-
J 1 PID (PID.ACK)
J 2 PID
K 3 PID
J 4 PID
J 5 PID
K 6 PID
K 7 PID
K 8 PID
-
_ SE0
_ SE0
J END
>>> print(pp_packet(wrap_packet(handshake_packet(PID.ACK))))
----
KKKK 1 Sync
JJJJ 2 Sync
KKKK 3 Sync
JJJJ 4 Sync
KKKK 5 Sync
JJJJ 6 Sync
KKKK 7 Sync
KKKK 8 Sync
----
JJJJ 1 PID (PID.ACK)
JJJJ 2 PID
KKKK 3 PID
JJJJ 4 PID
JJJJ 5 PID
KKKK 6 PID
KKKK 7 PID
KKKK 8 PID
----
____ SE0
____ SE0
JJJJ END
>>> print(pp_packet(wrap_packet(handshake_packet(PID.ACK), cycles=10), cycles=10))
----------
KKKKKKKKKK 1 Sync
JJJJJJJJJJ 2 Sync
KKKKKKKKKK 3 Sync
JJJJJJJJJJ 4 Sync
KKKKKKKKKK 5 Sync
JJJJJJJJJJ 6 Sync
KKKKKKKKKK 7 Sync
KKKKKKKKKK 8 Sync
----------
JJJJJJJJJJ 1 PID (PID.ACK)
JJJJJJJJJJ 2 PID
KKKKKKKKKK 3 PID
JJJJJJJJJJ 4 PID
JJJJJJJJJJ 5 PID
KKKKKKKKKK 6 PID
KKKKKKKKKK 7 PID
KKKKKKKKKK 8 PID
----------
__________ SE0
__________ SE0
JJJJJJJJJJ END

>>> print(pp_packet(wrap_packet(token_packet(PID.SETUP, 0, 0))))
----
KKKK 1 Sync
JJJJ 2 Sync
KKKK 3 Sync
JJJJ 4 Sync
KKKK 5 Sync
JJJJ 6 Sync
KKKK 7 Sync
KKKK 8 Sync
----
KKKK 1 PID (PID.SETUP)
JJJJ 2 PID
JJJJ 3 PID
JJJJ 4 PID
KKKK 5 PID
KKKK 6 PID
JJJJ 7 PID
KKKK 8 PID
----
JJJJ 1 Address
KKKK 2 Address
JJJJ 3 Address
KKKK 4 Address
JJJJ 5 Address
KKKK 6 Address
JJJJ 7 Address
KKKK 1 Endpoint
----
JJJJ 2 Endpoint
KKKK 3 Endpoint
JJJJ 4 Endpoint
KKKK 1 CRC5
KKKK 2 CRC5
JJJJ 3 CRC5
KKKK 4 CRC5
JJJJ 5 CRC5
----
____ SE0
____ SE0
JJJJ END
>>> print(pp_packet(wrap_packet(data_packet(PID.DATA0, [5, 6]))))
----
KKKK 1 Sync
JJJJ 2 Sync
KKKK 3 Sync
JJJJ 4 Sync
KKKK 5 Sync
JJJJ 6 Sync
KKKK 7 Sync
KKKK 8 Sync
----
KKKK 1 PID (PID.DATA0)
KKKK 2 PID
JJJJ 3 PID
KKKK 4 PID
JJJJ 5 PID
KKKK 6 PID
KKKK 7 PID
KKKK 8 PID
----
KKKK
JJJJ
JJJJ
KKKK
JJJJ
KKKK
JJJJ
KKKK
----
JJJJ
JJJJ
JJJJ
KKKK
JJJJ
KKKK
JJJJ
KKKK
----
KKKK  1 CRC16
JJJJ  2 CRC16
JJJJ  3 CRC16
JJJJ  4 CRC16
JJJJ  5 CRC16
JJJJ  6 CRC16
JJJJ  7 CRC16
KKKK  8 CRC16
----
KKKK  9 CRC16
JJJJ 10 CRC16
JJJJ 11 CRC16
JJJJ 12 CRC16
JJJJ 13 CRC16
KKKK 14 CRC16
JJJJ 15 CRC16
KKKK 16 CRC16
----
____ SE0
____ SE0
JJJJ END

>>> # Requires bit stuffing!
>>> print(pp_packet(wrap_packet(data_packet(PID.DATA0, [0x1]))))
----
KKKK 1 Sync
JJJJ 2 Sync
KKKK 3 Sync
JJJJ 4 Sync
KKKK 5 Sync
JJJJ 6 Sync
KKKK 7 Sync
KKKK 8 Sync
----
KKKK 1 PID (PID.DATA0)
KKKK 2 PID
JJJJ 3 PID
KKKK 4 PID
JJJJ 5 PID
KKKK 6 PID
KKKK 7 PID
KKKK 8 PID
----
KKKK
JJJJ
KKKK
JJJJ
KKKK
JJJJ
KKKK
JJJJ
----
JJJJ  1 CRC16
KKKK  2 CRC16
JJJJ  3 CRC16
KKKK  4 CRC16
JJJJ  5 CRC16
KKKK  6 CRC16
JJJJ  7 CRC16
JJJJ  8 CRC16
----
JJJJ  9 CRC16
JJJJ 10 CRC16
JJJJ 11 CRC16
JJJJ 12 CRC16
JJJJ 13 CRC16
KKKK    Bitstuff
KKKK 14 CRC16
KKKK 15 CRC16
JJJJ 16 CRC16
----
____ SE0
____ SE0
JJJJ END

>>> print(pp_packet(wrap_packet(data_packet(PID.DATA0, [0x1]))[:96]))
----
KKKK 1 Sync
JJJJ 2 Sync
KKKK 3 Sync
JJJJ 4 Sync
KKKK 5 Sync
JJJJ 6 Sync
KKKK 7 Sync
KKKK 8 Sync
----
KKKK 1 PID (PID.DATA0)
KKKK 2 PID
JJJJ 3 PID
KKKK 4 PID
JJJJ 5 PID
KKKK 6 PID
KKKK 7 PID
KKKK 8 PID
----
KKKK
JJJJ
KKKK
JJJJ
KKKK
JJJJ
KKKK
JJJJ END
>>> print(pp_packet(wrap_packet(sof_packet(12))))
----
KKKK 1 Sync
JJJJ 2 Sync
KKKK 3 Sync
JJJJ 4 Sync
KKKK 5 Sync
JJJJ 6 Sync
KKKK 7 Sync
KKKK 8 Sync
----
KKKK 1 PID (PID.SOF)
JJJJ 2 PID
JJJJ 3 PID
KKKK 4 PID
JJJJ 5 PID
JJJJ 6 PID
KKKK 7 PID
KKKK 8 PID
----
KKKK  1 Frame #
JJJJ  2 Frame #
KKKK  3 Frame #
JJJJ  4 Frame #
KKKK  5 Frame #
JJJJ  6 Frame #
KKKK  7 Frame #
JJJJ  8 Frame #
----
JJJJ  9 Frame #
JJJJ 10 Frame #
KKKK 11 Frame #
KKKK 1 CRC5
JJJJ 2 CRC5
KKKK 3 CRC5
JJJJ 4 CRC5
JJJJ 5 CRC5
----
____ SE0
____ SE0
JJJJ END

usbcore.utils.sdiff module

Whats sdiff.py

Compare two text files or directories; generate the resulting delta.

License

The MIT License (MIT)

usbcore.utils.sdiff.getcolor(withcolor, tag, side, openclose, isdircmp=False, withbg=None)[source]
usbcore.utils.sdiff.is_text(filepath)[source]
usbcore.utils.sdiff.strwidth(text, ambiguous_wide=True)[source]

A function to give back the width (a character width) of string.

Unit of width is one ASCII displayed by a monospaced font (This function is for environment using a wide character for)

Example:

>>> strwidth('teststring')
10
usbcore.utils.sdiff.expandtabs(text, tabsize=8, expandto='\t')[source]

Expand tabs(supports multibytes chars)

Example:

>>> expandtabs('text')
'text'

>>> expandtabs('\ta\tab\tend')
'\t\t\t\t\t\t\t\ta\t\t\t\t\t\t\tab\t\t\t\t\t\tend'

>>> expandtabs('abcdabc\tabcdabcd\tabcdabcda\tend')
'abcdabc\tabcdabcd\t\t\t\t\t\t\t\tabcdabcda\t\t\t\t\t\t\tend'

>>> expandtabs('\ta\tab\tabc\tabcd\tend', tabsize=4, expandto='@')
'@@@@a@@@ab@@abc@abcd@@@@end'
usbcore.utils.sdiff.strwidthdiv(text, width=180)[source]

divide string by appointed width

Example:

>>> strwidthdiv('teststring', 2)
['te', 'st', 'st', 'ri', 'ng']

>>> strwidthdiv('teststring', 3)
['tes', 'tst', 'rin', 'g']

>>> strwidthdiv('teststring', 8)
['teststri', 'ng']

>>> strwidthdiv('teststring', 15)
['teststring']
usbcore.utils.sdiff.strwidthdivsync(textarray, width=180)[source]

synclonize divide some string by appointed width

Example:

>>> strwidthdivsync(('test', 'string', ''), width=2)
[['te', 'st', ''], ['st', 'ri', 'ng'], ['', '', '']]

>>> strwidthdivsync(('test', 'string', ''), width=3)
[['tes', 't'], ['str', 'ing'], ['', '']]
class usbcore.utils.sdiff.unidiff[source]

Bases: object

Classes used by the unified diff parser to keep the diff data and Unified diff parser.

LINE_TYPE_ADD = '+'
LINE_TYPE_DELETE = '-'
LINE_TYPE_CONTEXT = ' '
class Hunk(src_start=0, src_len=0, tgt_start=0, tgt_len=0, section_header='')[source]

Bases: object

Each of the modified blocks of a file.

as_unified_diff()[source]

Output hunk data in unified diff format.

is_valid()[source]

Check hunk header data matches entered lines info.

append_context_line(line)[source]

Add a new context line to the hunk.

append_added_line(line)[source]

Add a new added line to the hunk.

append_deleted_line(line)[source]

Add a new deleted line to the hunk.

add_to_modified_counter(mods)[source]

Update the number of lines modified in the hunk.

class PatchedFile(source='', target='')[source]

Bases: list

Data of a patched file, each element is a Hunk.

as_unified_diff()[source]

Output file changes in unified diff format.

path

Return the file path abstracted from VCS.

added

Return the file total added lines.

deleted

Return the file total deleted lines.

modified

Return the file total modified lines.

is_added_file

Return True if this patch adds a file.

is_deleted_file

Return True if this patch deletes a file.

is_modified_file()[source]

Return True if this patch modifies a file.

class PatchSet[source]

Bases: list

A list of PatchedFiles.

as_unified_diff()[source]

Output patch data in unified diff format.

It won’t necessarily match the original unified diff, but it should be equivalent.

RE_SOURCE_FILENAME = re.compile('^--- (?P<filename>[^\\t\\n]+)')
RE_TARGET_FILENAME = re.compile('^\\+\\+\\+ (?P<filename>[^\\t\\n]+)')
RE_HUNK_HEADER = re.compile('^@@ -(\\d+)(?:,(\\d+))? \\+(\\d+)(?:,(\\d+))?\\ @@[ ]?(.*)')
RE_HUNK_BODY_LINE = re.compile('^([- \\+\\\\])')
exception UnidiffParseException[source]

Bases: Exception

Exception when parsing the diff data.

static parse_unidiff(diff)[source]

Unified diff parser, takes a file-like object as argument.

class usbcore.utils.sdiff.Differ(linejunk=None, charjunk=None, cutoff=0.75, fuzzy=0.0, cutoffchar=False, context=3)[source]

Bases: object

Differ is a class for comparing sequences of lines of text, and producing human-readable differences or deltas.

Differ uses SequenceMatcher both to compare sequences of lines, and to compare sequences of characters within similar (near-matching) lines.

Example:

>>> text1 = '''  1. Beautiful is better than ugly.
...   2. Explicit is better than implicit.
...   3. Simple is better than complex.
...   4. Complex is better than complicated.
... '''.splitlines(1)
>>>
>>> text2 = '''  1. Beautiful is better than ugly.
...   3.   Simple is better than complex.
...   4. Complicated is better than complex.
...   5. Flat is better than nested.
... '''.splitlines(1)
>>>
>>> d = Differ()
>>>
>>> result = list(d.compare(text1, text2))
>>>
>>> import pprint
>>> pprint.pprint(result, width=120)
[((' ', 0, '  1. Beautiful is better than ugly.\n', 0, '  1. Beautiful is better than ugly.\n'), None),
 (('<', 1, '  2. Explicit is better than implicit.\n', None, None), None),
 (('|', 2, '  3. Simple is better than complex.\n', 1, '  3.   Simple is better than complex.\n'),
  [(' ', '  3.', '  3.'),
   ('+', None, '  '),
   (' ', ' Simple is better than complex.\n', ' Simple is better than complex.\n')]),
 (('|', 3, '  4. Complex is better than complicated.\n', 2, '  4. Complicated is better than complex.\n'),
  [(' ', '  4. Compl', '  4. Compl'),
   ('+', None, 'icat'),
   (' ', 'e', 'e'),
   ('!', 'x', 'd'),
   (' ', ' is better than compl', ' is better than compl'),
   ('-', 'icat', None),
   (' ', 'e', 'e'),
   ('!', 'd', 'x'),
   (' ', '.\n', '.\n')]),
 (('>', None, None, 3, '  5. Flat is better than nested.\n'), None),
 None]
compare(text1, text2)[source]

Compare two sequences of lines; generate the resulting delta.

Example:

>>> text1 = '''one
... two
... three
... '''.splitlines(1)
>>>
>>> text2 = '''ore
... tree
... emu
... '''.splitlines(1)
>>>
>>> import pprint
>>>
>>> pprint.pprint(list(Differ().compare(text1, text2)), width=100)
[(('>', None, None, 0, 'ore\n'), None),
 (('<', 0, 'one\n', None, None), None),
 (('<', 1, 'two\n', None, None), None),
 (('|', 2, 'three\n', 1, 'tree\n'), [(' ', 't', 't'), ('-', 'h', None), (' ', 'ree\n', 'ree\n')]),
 (('>', None, None, 2, 'emu\n'), None),
 None]
>>>
>>> # like sdiff
>>> pprint.pprint(list(Differ(cutoff=0, fuzzy=1).compare(text1, text2)), width=100)
[(('|', 0, 'one\n', 0, 'ore\n'), [(' ', 'o', 'o'), ('!', 'n', 'r'), (' ', 'e\n', 'e\n')]),
 (('|', 1, 'two\n', 1, 'tree\n'), [(' ', 't', 't'), ('!', 'wo', 'ree'), (' ', '\n', '\n')]),
 (('|', 2, 'three\n', 2, 'emu\n'),
  [('-', 'thr', None), (' ', 'e', 'e'), ('!', 'e', 'mu'), (' ', '\n', '\n')]),
 None]
static formattext(tag, num1, text1, num2, text2, width, withcolor=False, linediff=None)[source]

Example:

>>> Differ.formattext('|', 1, 'aaa', 2, 'bbb', 80)
['     2|aaa                             |      3|bbb']

>>> Differ.formattext('|', 1, 'aaa', 2, 'bbb', 60)
['     2|aaa                   |      3|bbb']

>>> Differ.formattext(' ', 1, 'aaa', 2, 'aaa', 80)
['     2|aaa                                    3|aaa']

>>> Differ.formattext('<', 1, 'aaa', None, None, 80)
['     2|aaa                             <       |']

>>> Differ.formattext('>', None, None, 2, 'bbb', 80)
['      |                                >      3|bbb']

>>> import pprint
>>> pprint.pprint(Differ.formattext('>',
...                                 1, 'a' * 60,
...                                 2, 'b' * 20, 60))
['     2|aaaaaaaaaaaaaaaaaaaaa >      3|bbbbbbbbbbbbbbbbbbbb',
 '     ^|aaaaaaaaaaaaaaaaaaaaa ^       |',
 '     ^|aaaaaaaaaaaaaaaaaa    ^       |']
static formatlinetext(num1, num2, linediff, width, withcolor=False)[source]

Example:

>>> import pprint
>>> pprint.pprint(Differ.formatlinetext(1, 2,
...                                     [('!', 'bbb', 'aaaaa'),
...                                      (' ', 'cc', 'cc'),
...                                      ('+', None, 'dd'),
...                                      ('-', 'ee', None)], 80))
['[     ]      |!!!++  ++--',
 '[ <-  ]     2|bbb  cc  ee',
 '[  -> ]     3|aaaaaccdd  ']
class usbcore.utils.sdiff.ext_dircmp(a, b, ignore=None, hide=None)[source]

Bases: filecmp.dircmp

phase1()[source]
phase2()[source]
phase3()[source]
phase4()[source]
dirtree()[source]
usbcore.utils.sdiff.formatdircmp(tag, head1, text1, head2, text2, width, cont_mark1='^', cont_mark2='^', sep_mark='|', withcolor=False)[source]
usbcore.utils.sdiff.original_diff(lines1, lines2, linejunk, charjunk, cutoff, fuzzy, cutoffchar, context, width, withcolor=False)[source]

Example:

>>> text1 = '''  1. Beautiful is better than ugly.
...   2. Explicit is better than implicit.
...   3. Simple is better than complex.
...   4. Complex is better than complicated.
... '''.splitlines(1)
>>>
>>> text2 = '''  1. Beautiful is better than ugly.
...   3.   Simple is better than complex.
...   4. Complicated is better than complex.
...   5. Flat is better than nested.
... '''.splitlines(1)
>>>
>>> diff = original_diff(text1, text2, linejunk=None, charjunk=None,
...                      cutoff=0.1, fuzzy=0,
...                      cutoffchar=False, context=5,
...                      width=100)
>>> for line in diff: print('\'' + line + '\'')
'     1|  1. Beautiful is better than ugly.              1|  1. Beautiful is better than ugly.'
'     2|  2. Explicit is better than implicit.    <       |'
'     3|  3. Simple is better than complex.       |      2|  3.   Simple is better than complex.'
'     4|  4. Complex is better than complicated.  |      3|  4. Complicated is better than complex.'
'      |                                          >      4|  5. Flat is better than nested.'
''
'[     ]      |    ++                                '
'[ <-  ]     3|  3.   Simple is better than complex. '
'[  -> ]     2|  3.   Simple is better than complex. '
''
'[     ]      |          ++++ !                     ---- !  '
'[ <-  ]     4|  4. Compl    ex is better than complicated. '
'[  -> ]     3|  4. Complicated is better than compl    ex. '
''
usbcore.utils.sdiff.dircmp(dir1, dir2, enc_filepath='utf-8', recursive=False)[source]

Compare directories.

usbcore.utils.sdiff.parse_unidiff(diff)[source]

Unified diff parser, takes a file-like object as argument.

Example:

>>> hg_diff = r'''diff -r dab26450e4b1 text2.txt
... --- a/text2.txt     Sun Dec 15 17:38:49 2013 +0900
... +++ b/text2.txt     Sun Dec 15 17:43:09 2013 +0900
... @@ -1,3 +1,3 @@
... -hoge
... +hogee
... +bar
...  foo
... -bar
... '''
>>> diffs = parse_unidiff((line for line in hg_diff.splitlines()))
>>> for (flag, diff) in diffs:
...     if flag: print(diff)
...     else:
...         for hunk in diff:
...             import pprint
...             pprint.pprint(hunk.source_lines)
...             pprint.pprint(hunk.target_lines)
...
diff -r dab26450e4b1 text2.txt
--- a/text2.txt     Sun Dec 15 17:38:49 2013 +0900
+++ b/text2.txt     Sun Dec 15 17:43:09 2013 +0900
['hoge', 'foo', 'bar']
['hogee', 'bar', 'foo']
>>>
usbcore.utils.sdiff.parse_unidiff_and_original_diff(udiffs, linejunk, charjunk, cutoff, fuzzy, cutoffchar, context, width, withcolor=False)[source]

Example:

>>> svn_diff = u'''Index: some.png
... ===================================================================
... Cannot display: file marked as a binary type.
... svn:mime-type = application/octet-stream
... Index: text1.txt
... ===================================================================
... --- text1.txt       (revision 1)
... +++ text1.txt       (working copy)
... @@ -1,4 +1,4 @@
...  1. Beautiful is better than ugly.
... -2. Explicit is better than implicit.
... -3. Simple is better than complex.
... -4. Complex is better than complicated.
... +3.   Simple is better than complex.
... +4. Complicated is better than complex.
... +5. Flat is better than nested.
... '''
>>> diff = parse_unidiff_and_original_diff(
...     (line for line in svn_diff.splitlines()),
...     linejunk=None, charjunk=None,
...     cutoff=0.1, fuzzy=0,
...     cutoffchar=False, context=5,
...     width=100)
>>> for line in diff: print('\'' + line + '\'')
'Index: some.png'
'==================================================================='
'Cannot display: file marked as a binary type.'
'svn:mime-type = application/octet-stream'
'Index: text1.txt'
'==================================================================='
'--- text1.txt       (revision 1)'
'+++ text1.txt       (working copy)'
'     1|1. Beautiful is better than ugly.                1|1. Beautiful is better than ugly.'
'     2|2. Explicit is better than implicit.      <       |'
'     3|3. Simple is better than complex.         |      2|3.   Simple is better than complex.'
'     4|4. Complex is better than complicated.    |      3|4. Complicated is better than complex.'
'      |                                          >      4|5. Flat is better than nested.'
''
'[     ]      |  ++                               '
'[ <-  ]     3|3.   Simple is better than complex.'
'[  -> ]     2|3.   Simple is better than complex.'
''
'[     ]      |        ++++ !                     ---- ! '
'[ <-  ]     4|4. Compl    ex is better than complicated.'
'[  -> ]     3|4. Complicated is better than compl    ex.'
''
>>>
>>> hg_diff = u'''diff -r dab26450e4b1 some.png
... Binary file some.png has changed
... diff -r dab26450e4b1 text1.txt
... --- a/text1.txt     Sun Dec 15 17:38:49 2013 +0900
... +++ b/text1.txt     Sun Dec 15 17:43:09 2013 +0900
... @@ -1,4 +1,4 @@
...  1. Beautiful is better than ugly.
... -2. Explicit is better than implicit.
... -3. Simple is better than complex.
... -4. Complex is better than complicated.
... +3.   Simple is better than complex.
... +4. Complicated is better than complex.
... +5. Flat is better than nested.
... '''
>>> diff = parse_unidiff_and_original_diff(
...     (line for line in hg_diff.splitlines()),
...     linejunk=None, charjunk=None,
...     cutoff=0.1, fuzzy=0,
...     cutoffchar=False, context=5,
...     width=100)
>>> for line in diff: print('\'' + line + '\'')
'diff -r dab26450e4b1 some.png'
'Binary file some.png has changed'
'diff -r dab26450e4b1 text1.txt'
'--- a/text1.txt     Sun Dec 15 17:38:49 2013 +0900'
'+++ b/text1.txt     Sun Dec 15 17:43:09 2013 +0900'
'     1|1. Beautiful is better than ugly.                1|1. Beautiful is better than ugly.'
'     2|2. Explicit is better than implicit.      <       |'
'     3|3. Simple is better than complex.         |      2|3.   Simple is better than complex.'
'     4|4. Complex is better than complicated.    |      3|4. Complicated is better than complex.'
'      |                                          >      4|5. Flat is better than nested.'
''
'[     ]      |  ++                               '
'[ <-  ]     3|3.   Simple is better than complex.'
'[  -> ]     2|3.   Simple is better than complex.'
''
'[     ]      |        ++++ !                     ---- ! '
'[ <-  ]     4|4. Compl    ex is better than complicated.'
'[  -> ]     3|4. Complicated is better than compl    ex.'
''
>>>
usbcore.utils.sdiff.getTerminalSize()[source]
usbcore.utils.sdiff.getdefaultencoding()[source]
usbcore.utils.sdiff.main()[source]

main function

usbcore.utils.vcd module

usbcore.utils.vcd.write_gtkwave_file(vcd_filename)[source]
usbcore.utils.vcd.add_vcd_timescale(filename, timescale=435)[source]

Module contents