Package org.jsoup.parser

Class Tokeniser

java.lang.Object
org.jsoup.parser.Tokeniser
final class Tokeniser extends Object
Readers the input stream into tokens.

  • Field Details

    • replacementChar

      static final char replacementChar
      See Also:

    • notCharRefCharsSorted

      private static final char[] notCharRefCharsSorted

    • win1252ExtensionsStart

      static final int win1252ExtensionsStart
      See Also:

    • win1252Extensions

      static final int[] win1252Extensions

    • reader

      private final CharacterReader reader

    • errors

      private final ParseErrorList errors

    • state

      private TokeniserState state

    • emitPending

      private Token emitPending

    • isEmitPending

      private boolean isEmitPending

    • dataBuffer

      final TokenData dataBuffer

    • syntax

      final Document.OutputSettings.Syntax syntax

    • startPending

      final Token.StartTag startPending

    • endPending

      final Token.EndTag endPending

    • tagPending

      Token.Tag tagPending

    • charPending

      final Token.Character charPending

    • doctypePending

      final Token.Doctype doctypePending

    • commentPending

      final Token.Comment commentPending

    • xmlDeclPending

      final Token.XmlDecl xmlDeclPending

    • lastStartTag

      private String lastStartTag

    • lastStartCloseSeq

      private String lastStartCloseSeq

    • markupStartPos

      private int markupStartPos

    • charStartPos

      private int charStartPos

    • codepointHolder

      private final int[] codepointHolder

    • multipointHolder

      private final int[] multipointHolder

  • Constructor Details

  • Method Details

    • read

      Token read()

    • emit

      void emit(Token token)

    • emit

      void emit(String str)

    • emit

      void emit(char c)

    • emit

      void emit(int[] codepoints)

    • transition

      void transition(TokeniserState newState)

    • advanceTransition

      void advanceTransition(TokeniserState newState)

    • consumeCharacterReference

      int[] consumeCharacterReference(Character additionalAllowedCharacter, boolean inAttribute)
      Tries to consume a character reference, and returns: null if nothing, int[1], or int[2].

    • createTagPending

      Token.Tag createTagPending(boolean start)

    • createXmlDeclPending

      Token.XmlDecl createXmlDeclPending(boolean isDeclaration)

    • emitTagPending

      void emitTagPending()

    • createCommentPending

      void createCommentPending()

    • emitCommentPending

      void emitCommentPending()

    • createBogusCommentPending

      void createBogusCommentPending()

    • createDoctypePending

      void createDoctypePending()

    • emitDoctypePending

      void emitDoctypePending()

    • createTempBuffer

      void createTempBuffer()

    • isAppropriateEndTagToken

      boolean isAppropriateEndTagToken()

    • appropriateEndTagName

      String appropriateEndTagName()

    • appropriateEndTagSeq

      String appropriateEndTagSeq()
      Returns the closer sequence </lastStart

    • error

      void error(TokeniserState state)

    • eofError

      void eofError(TokeniserState state)

    • characterReferenceError

      private void characterReferenceError(String message, Object... args)

    • error

      void error(String errorMsg)

    • error

      void error(String errorMsg, Object... args)

    • unescapeEntities

      String unescapeEntities(boolean inAttribute)
      Utility method to consume reader and unescape entities found within.
      Parameters:
      inAttribute - if the text to be unescaped is in an attribute
      Returns:
      unescaped string from reader