github.com/sercand/please@v13.4.0+incompatible/docs/advanced.html (about)

     1  
     2      <h1>Advanced Please</h1>
     3  
     4      <p>So after reading the <a href="basics.html">rest</a> <a href="intermediate.html">of it</a>, you're wondering what else it can do?</p>
     5      <p>This section covers the more esoteric bits of Please functionality. If you're familiar with Blaze or Buck,
     6        these probably still won't be familiar :)</p>
     7  
     8      <h2>Pre- and post-build actions</h2>
     9  
    10      <p>It's possible in Please to define callbacks in Python that are invoked either immediately before or immediately after
    11        the target is built. These allow modifying aspects of the target or the wider build graph which can potentially
    12        be a powerful tool to handle things that are otherwise awkward or impossible.</p>
    13  
    14      <p>Note that these functions are only evaluated at build time, so their results will not be visible to <code>plz query</code>
    15        and they can be a little hard to debug if you get things wrong. They should hence be used judiciously.</p>
    16  
    17      <h3>Pre-build function</h3>
    18  
    19      <p>The pre-build function is defined on a <code>build_rule</code> as simply <code>pre_build = lambda name: do_stuff(name)</code>.</p>
    20  
    21      <p>As you can see, it's invoked with one argument, the name of the rule about to be built. It's up to you what you do in
    22        the function, although in practice the only really useful thing at present is to inspect the rule's transitive labels and
    23        adjust its build command accordingly. This is done via calling <code>get_labels(name, prefix)</code> where <code>name</code>
    24        is the name of the current rule and <code>prefix</code> is a prefix the labels you're interested in will have (for
    25        convenience, it's stripped from the returned values). Having got these, one can then call <code>set_command(name, cmd)</code>
    26        to alter the command for your rule.</p>
    27  
    28      <p>The built-in C++ rules for Please (<code><a href="https://github.com/thought-machine/please/blob/master/src/parse/rules/cc_rules.build_defs">src/parse/rules/cc_rules.build_defs</a></code>) are a reasonably good example of how to use this.</p>
    29  
    30      <h3>Post-build function</h3>
    31  
    32      <p>The post-build function is somewhat more powerful and useful than the pre-build function, and hence it appeared in the
    33        API several months sooner. It is defined similarly (<code>post_build = lambda name, output: do_stuff(output)</code>) but
    34        takes an extra output which is the standard output of the build rule after successful invocation.</p>
    35  
    36      <p>The power of this is that you can run a build rule to do arbitrary operations and then alter the build graph based
    37        on that; currently you can add outputs to the rule, define new rules and add dependencies to a rule. These lead to
    38        two motivating examples which both occur in built-in rules:</p>
    39  
    40      <p>First, collecting additional output files. This happens with Java protobuf outputs where the output files are not obvious
    41        until the .proto file is parsed, because their location is defined by the <code>option java_package</code> stanza. The
    42        first iteration of <code>proto_library</code> required these to be explicitly defined but that rapidly became tedious;
    43        the nicer solution is detecting it this way. The build rule simply invokes <code>find</code> to locate all the .java files
    44        it produces and the post-build function receives those and runs <code>add_out(name, java_file)</code> for each.</p>
    45  
    46      <p>Second, we wanted to be able to fetch third-party dependencies without having to expand a transitive dependency tree
    47        into separate <code>maven_jar</code> rules by hand. <code>maven_jars</code> does this by hitting up Maven to get the
    48        dependencies then generating a new build rule for each.</p>
    49  
    50      <h2>Require / Provide</h2>
    51  
    52      <p>This is a generalised mechanism for rules to provide specialised dependencies to other rules. The most obvious example
    53        is again <code>proto_library</code>, also illustrating how it's a harder rule to write well than you'd think.</p>
    54  
    55      <p>The problem with <code>proto_library</code> is that we ideally want to maintain the illusion that there's one rule
    56        by the name given in the BUILD file and have all other rules depend only on that, but of course it generates outputs
    57        for multiple languages and it's pretty suboptimal to have your <code>python_library</code> have to wait for the
    58        C++ and Java generated code to compile.</p>
    59  
    60      <p>This is solved by the <code>proto_library</code> rule declaring the following property:
    61        <pre><code>
    62  	provides = {
    63  	    'py': ':python_dependency',
    64  	    'java': ':java_dependency',
    65              'cc': ':cc_dependency',
    66          }
    67        </code></pre>
    68        And then the <code>python_library</code> rule is annotated with
    69        <pre><code>
    70  	  requires = ['py']
    71        </code></pre>
    72        Individually neither has any effect, but when a rule with a particular <code>require</code> depends on another with
    73        a matching <code>provide</code>, its dependency is matched directly to the one specified by the providing rule. This
    74        means that it can skip other dependencies that it doesn't care about and be matched only to the ones it does.</p>
    75  
    76      <p>The consequence of this - skipping the actual declared dependency - can be a bit nonobvious so again should be
    77        invoked sparingly. Typically the rule declaring <code>provides</code> should be a no-op collecting rule such as
    78        <code>filegroup</code> and the various provided rules shouldn't normally be needed together. Different languages
    79        are the most common example here (and indeed this feature was nearly named <code>language</code> at one point).</p>
    80  
    81      <p>The <code>requires</code> stanza is also responsible for colouring the interactive build output. Currently the available
    82        colours are hardcoded but one of these days they might become configurable.</p>
    83  
    84      <h2>Hash verification</h2>
    85  
    86      <p>Please can natively verify hashes of packages. Some of the built-in rules for fetching things from third-party
    87        repos have this option, and you can add it to your own genrules. For example, one of the Python libraries we use:
    88  
    89        <pre><code>
    90          pip_library(
    91              name = 'six',
    92              version = '1.9.0',
    93              outs = ['six.py'],
    94              hashes = ['sha1: 0c31ab7cf1a2761efa32d9a7e891ddeadc0d8673'],
    95          )
    96        </code></pre>
    97  
    98        This declares that the calculated sha1 hash of the package must match one of the given set, and it's a failure
    99        if not.
   100      </p>
   101  
   102      <p>You can find the output hash of a particular target by running <code>plz hash //third_party/python:six</code>
   103        which will calculate it for you, and you can enter it in the BUILD file. At some point in the future we'd like
   104        to provide better tooling around this (eg. automatically updating the build file with a particular hash).</p>
   105  
   106      <p>The reason for allowing multiple hashes is for rules that generate different outputs on different architectures;
   107        this is common for Python libraries which have a compiled component, for example.</p>
   108  
   109      <p>For testing purposes you can run Please with the <code>--nohash_verification</code> flag which will reduce hash
   110        verification failures to a warning message only.</p>
   111  
   112      <p>Note that when using this you must be careful that the outputs of your rule are really deterministic. This is
   113        generally true for <code>remote_file</code> calls, but obviously only if the server returns the same thing
   114        every time for that URL. Some care should be taken with <code>pip_library</code> since the outputs of a
   115        'pip install' for a package containing binary (not pure Python) modules are not bit-for-bit identical
   116        if compiled locally, only if you downloaded a precompiled wheel. Different Python and OS versions can affect
   117        it too.</p>
   118  
   119      <h2>Licence validation</h2>
   120  
   121      <p>Please can attempt to autodetect licences from third-party packages and inform you if they're not ones you'd
   122        accept. You mark licences in the .plzconfig file like so:
   123        <pre><code>
   124          [licences]
   125          accept = MIT
   126          accept = BSD
   127          reject = MS-EULA
   128        </code></pre>
   129        By default, with no <code>[licences]</code> section, Please won't perform any licence checking.
   130        Once you've added some any package with a licence must have a matching accepted licence in the config.</p>
   131  
   132      <p>Currently we can autodetect licences from <code>pip_library</code> and <code>maven_jars</code> rules,
   133        you can also set them manually via the <code>licences</code> attribute on a rule.</p>
   134  
   135      <p>It bears mentioning that this is done as a best-effort - since licences and their locations are not standardised
   136        in pip (and many other places) we can't always be fully confident about this, so it can't be a full substitute for
   137        validating licences yourself.</p>