skip to content

In xml.sax, what do feature_external_ges and feature_external_pes control?

level: middleimportance: nice to knowfreq 14%

answer

  1. Feature URIs, not parser attributes
  2. General entities versus parameter entities
  3. One is a live switch, one is inert
  4. Off by default since a 3.7 patch release
  5. Enabling the parameter one raises SAXNotSupportedException

basics

~20 s

They are the SAX feature identifiers for external general and external parameter entities. Enabling feature_external_ges makes the reader fetch whatever an entity's system identifier points at; the expat-backed reader refuses any attempt to enable feature_external_pes.

solid answer

~40 s

Both are feature URIs you pass to a SAX reader's `setFeature`. `xml.sax.handler.feature_external_ges` controls external *general* entities — the `&xxe;` case. It has defaulted to `False` since Python 3.7.1, and flipping it to `True` makes the reader resolve the entity's system identifier through its `xml.sax.handler.EntityResolver`, so the contents of a `file://` path or an HTTP URL land directly in your content handler's character data. `xml.sax.handler.feature_external_pes` covers external *parameter* entities, referenced from within the DTD. The expat-backed reader never reads those: `getFeature` returns `False`, setting it to `False` is accepted as a no-op, and setting it to `True` raises `xml.sax.SAXNotSupportedException("expat does not read external parameter entities")`. So one flag is a real footgun you must leave alone, and the other cannot be turned on at all.

code

python · 29 lines
python
import io
import tempfile
import xml.sax
from xml.sax.handler import ContentHandler, feature_external_ges

with tempfile.NamedTemporaryFile("w", suffix=".txt", delete=False) as fh:
    fh.write("CONFIDENTIAL")
    secret = fh.name

doc = (f'<?xml version="1.0"?>'
       f'<!DOCTYPE d [<!ENTITY xxe SYSTEM "file://{secret}">]>'
       f'<d>&xxe;</d>').encode()


class Collect(ContentHandler):
    def __init__(self):
        self.text = []

    def characters(self, content):
        self.text.append(content)


for enabled in (False, True):
    parser = xml.sax.make_parser()
    handler = Collect()
    parser.setContentHandler(handler)
    parser.setFeature(feature_external_ges, enabled)
    parser.parse(io.BytesIO(doc))
    print(f"external_ges={enabled}: {''.join(handler.text)!r}")

go deeper

for a junior

Know that xml.sax parsers are configured with feature identifier strings, and that the one controlling external general entities is off by default — you should never need to switch it on.

for a middle

Explain what each feature covers, that enabling the general-entities one routes through an EntityResolver and pulls remote or local content into your character data, and that the parameter-entities one cannot be enabled at all.

for a senior

Show the limits of the flags: they say nothing about internal entity expansion, size or depth, so a hardening claim built only on them is empty. Point at a single sanctioned parse entry point instead.

for a principal

Treat the per-module inconsistency as the real problem — one front-end raises, one skips silently, one flag is unsettable — and standardise on one hardened parsing surface so no team has to know the matrix.

## SAX features are string identifiers, not attributes A reader built by `xml.sax.make_parser` is configured through `xml.sax.xmlreader.XMLReader.setFeature` and read back with `xml.sax.xmlreader.XMLReader.getFeature`, each taking a **feature URI**. `xml.sax.handler` defines those constants: - `feature_external_ges` is `http://xml.org/sax/features/external-general-entities`; - `feature_external_pes` is `http://xml.org/sax/features/external-parameter-entities`. Because they are plain strings, a typo is a runtime `SAXNotRecognizedException` rather than an attribute error — worth knowing when you are auditing someone else's hardening code. ## General entities: the flag that actually matters An external *general* entity is the one referenced from the document body: `<!ENTITY xxe SYSTEM "file:///secret">` plus `&xxe;` in an element. Before Python 3.7.1 the SAX reader resolved these by default. Since then the default is off, and the behaviour with the flag off is *quiet*: the parse completes, and the content handler simply receives no characters for that reference. Turn the flag on and the same document makes the reader fetch the system identifier through its `xml.sax.handler.EntityResolver` and hand you the bytes as ordinary character data. The demonstration is uncomfortably short — the same parse, the same document, one boolean, and a local file's contents appear in your text buffer. Teams do turn it on, usually by accident or by cargo cult, because some legitimate documents reference a shared external DTD and someone found that flipping the flag "made the parse work". That is the moment a document parser becomes a file reader and an outbound HTTP client, so the review rule is simple: **the flag stays off**, and documents that genuinely need a shared DTD get it resolved out of band from a location you control. ## Parameter entities: the flag you cannot enable An external *parameter* entity is referenced with `%name;` from inside the DTD, and it is the vehicle for the more advanced entity tricks, including out-of-band exfiltration of data via a crafted external DTD. Here Python is **structurally safe** rather than merely defaulted safe: libexpat does not read external parameter entities at all. - Ask the reader for the feature and you get `False`; - set it to `False` and the call is accepted and does nothing; - set it to `True` and you get `xml.sax.SAXNotSupportedException: expat does not read external parameter entities`. Code that hardens a parser by setting both features to `False` therefore works, but only because the second call happens to be a no-op — and code that tries to be thorough by asserting it could be enabled would crash. ## What the two flags do not buy you Neither one touches internal entity declarations. A document whose DTD subset declares nested entities that reference each other still expands through the SAX reader exactly as it does through `xml.etree.ElementTree`, and the only thing stopping it is libexpat's own amplification limit from version 2.4.1 onwards. Neither flag caps document size, nesting depth or the number of declarations. So a hardening story that consists of "we set the external-entity features to False" is describing the default and the impossible, and has addressed nothing that is actually still open. ## How this compares to the other front-ends - `xml.etree.ElementTree` exposes no equivalent switch at all — it simply raises `xml.etree.ElementTree.ParseError` on an undefined entity, which reaches the same place by a different route. - `xml.dom.pulldom` sits on the same SAX machinery and therefore inherits these features. That asymmetry is the practical reason a hardened third-party XML library exists: it gives every front-end one stated policy — no DTD, no entity declarations, a dedicated exception — instead of a per-module mix of defaults, silent skips and unsupported flags. ## How to answer it in an interview 1. Name both features, 2. say which one is a live risk and which is inert, 3. state that general entities have been off by default since 3.7.1, 4. and finish on the limitation: neither flag addresses internal entity expansion, which is the shape that does not need the network at all. ## Verify the setting; do not assume it Because features are identified by strings and readers differ in what they support, hardening code deserves a test: parse a document whose entity points at a file the test itself created, and assert that the content handler never saw its contents. That test also records intent for the next reader, who otherwise sees two `setFeature` calls with no way to tell which of them does anything. Note what such a test does not prove — it will pass identically on a build whose linked libexpat has no amplification limit, because these features govern external references only and say nothing about the cost of expanding entities the document declares itself.

  • What happens in xml.sax when the general-entities feature is left at its default and the document references an external entity?
    The parse succeeds and the content handler receives no characters for that reference. There is no exception and no marker in the tree of events, so the application sees an empty field. That silence is why the same hostile document produces a loud ParseError from xml.etree.ElementTree and a clean parse from xml.sax.
  • Is setting both features to False a complete hardening step?
    No. One of the two is already the default and the other cannot be enabled anyway, so the pair of calls changes nothing. Internal entity declarations still expand, and no document-size, depth or declaration-count limit is applied. Real hardening means refusing a DOCTYPE outright, which the SAX feature set has no way to express.
  • Why do the features take URI strings rather than being keyword arguments?
    SAX is a language-independent interface whose feature and property names are namespaced URIs, and Python's xml.sax mirrors it. The practical consequence is that an unknown name raises xml.sax.SAXNotRecognizedException at runtime, and an unsupported value raises xml.sax.SAXNotSupportedException — so hardening code should be exercised by a test rather than assumed to have taken effect.

saying these in an interview costs you the question

  • Confuses general entities with parameter entities
  • Claims external general entities are on by default
  • Thinks setting both features off is complete hardening
  • Expects a parse error when an external entity is skipped
  • Assumes the features also bound expansion size or depth

context

resolving links…