github.com/RedHatInsights/insights-results-aggregator@v1.4.1/docs/json_check.html (about) 1 <!DOCTYPE html> 2 <html> 3 <head> 4 <meta http-equiv="content-type" content="text/html;charset=utf-8"> 5 <title>json_check.py</title> 6 <link rel="stylesheet" href="pycco.css"> 7 </head> 8 <body> 9 <div id='container'> 10 <div id="background"></div> 11 <div class='section'> 12 <div class='docs'><h1>json_check.py</h1></div> 13 </div> 14 <div class='clearall'> 15 <div class='section' id='section-0'> 16 <div class='docs'> 17 <div class='octowrap'> 18 <a class='octothorpe' href='#section-0'>#</a> 19 </div> 20 <p>Copyright © 2020 Pavel Tisnovsky</p> 21 <p>Licensed under the Apache License, Version 2.0 (the “License”); 22 you may not use this file except in compliance with the License. 23 You may obtain a copy of the License at</p> 24 <pre><code>http://www.apache.org/licenses/LICENSE-2.0 25 </code></pre> 26 <p>Unless required by applicable law or agreed to in writing, software 27 distributed under the License is distributed on an “AS IS” BASIS, 28 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 29 See the License for the specific language governing permissions and 30 limitations under the License.</p> 31 </div> 32 <div class='code'> 33 <div class="highlight"><pre><span></span></pre></div> 34 </div> 35 </div> 36 <div class='clearall'></div> 37 <div class='section' id='section-1'> 38 <div class='docs'> 39 <div class='octowrap'> 40 <a class='octothorpe' href='#section-1'>#</a> 41 </div> 42 <p>Simple checker of all JSONs in the given directory (usually repository).</p> 43 <h2>Usage:</h2> 44 <pre><code>json_check.py [-h] [-v] [-n] -d DIRECTORY 45 </code></pre> 46 <h2>Optional arguments:</h2> 47 <pre><code>-h, --help show this help message and exit 48 -v, --verbose make it verbose 49 -n, --no-colors disable color output 50 -d DIRECTORY--directory DIRECTORY 51 directory with JSON files to check 52 </code></pre> 53 </div> 54 <div class='code'> 55 <div class="highlight"><pre></pre></div> 56 </div> 57 </div> 58 <div class='clearall'></div> 59 <div class='section' id='section-2'> 60 <div class='docs'> 61 <div class='octowrap'> 62 <a class='octothorpe' href='#section-2'>#</a> 63 </div> 64 <h2>Link to generated documentation for this script:</h2> 65 <p><a href="https://redhatinsights.github.io/insights-results-aggregator/json_check.html">https://redhatinsights.github.io/insights-results-aggregator/json_check.html</a></p> 66 </div> 67 <div class='code'> 68 <div class="highlight"><pre><span class="kn">from</span> <span class="nn">pathlib</span> <span class="kn">import</span> <span class="n">Path</span> 69 <span class="kn">from</span> <span class="nn">json</span> <span class="kn">import</span> <span class="n">load</span> 70 <span class="kn">from</span> <span class="nn">sys</span> <span class="kn">import</span> <span class="nb">exit</span> 71 <span class="kn">from</span> <span class="nn">os</span> <span class="kn">import</span> <span class="n">popen</span> 72 <span class="kn">from</span> <span class="nn">argparse</span> <span class="kn">import</span> <span class="n">ArgumentParser</span></pre></div> 73 </div> 74 </div> 75 <div class='clearall'></div> 76 <div class='section' id='section-3'> 77 <div class='docs'> 78 <div class='octowrap'> 79 <a class='octothorpe' href='#section-3'>#</a> 80 </div> 81 <p>Try to execute tput to read control code for selected operation.</p> 82 </div> 83 <div class='code'> 84 <div class="highlight"><pre><span class="k">def</span> <span class="nf">read_control_code</span><span class="p">(</span><span class="n">operation</span><span class="p">):</span></pre></div> 85 </div> 86 </div> 87 <div class='clearall'></div> 88 <div class='section' id='section-4'> 89 <div class='docs'> 90 <div class='octowrap'> 91 <a class='octothorpe' href='#section-4'>#</a> 92 </div> 93 94 </div> 95 <div class='code'> 96 <div class="highlight"><pre> <span class="k">return</span> <span class="n">popen</span><span class="p">(</span><span class="s2">"tput "</span> <span class="o">+</span> <span class="n">operation</span><span class="p">,</span> <span class="s2">"r"</span><span class="p">)</span><span class="o">.</span><span class="n">readline</span><span class="p">()</span></pre></div> 97 </div> 98 </div> 99 <div class='clearall'></div> 100 <div class='section' id='section-5'> 101 <div class='docs'> 102 <div class='octowrap'> 103 <a class='octothorpe' href='#section-5'>#</a> 104 </div> 105 <p>Check all JSON files found in current directory and all subdirectories.</p> 106 </div> 107 <div class='code'> 108 <div class="highlight"><pre><span class="k">def</span> <span class="nf">check_jsons</span><span class="p">(</span><span class="n">verbose</span><span class="p">,</span> <span class="n">directory</span><span class="p">):</span></pre></div> 109 </div> 110 </div> 111 <div class='clearall'></div> 112 <div class='section' id='section-6'> 113 <div class='docs'> 114 <div class='octowrap'> 115 <a class='octothorpe' href='#section-6'>#</a> 116 </div> 117 <p>Reset counters with number of passes and number of failures.</p> 118 </div> 119 <div class='code'> 120 <div class="highlight"><pre> <span class="n">passes</span> <span class="o">=</span> <span class="mi">0</span> 121 <span class="n">failures</span> <span class="o">=</span> <span class="mi">0</span></pre></div> 122 </div> 123 </div> 124 <div class='clearall'></div> 125 <div class='section' id='section-7'> 126 <div class='docs'> 127 <div class='octowrap'> 128 <a class='octothorpe' href='#section-7'>#</a> 129 </div> 130 <p>Find all files in current directory and subdirectories with <code>*.json</code> 131 extension. Files are found recursivelly.</p> 132 </div> 133 <div class='code'> 134 <div class="highlight"><pre> <span class="n">files</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">Path</span><span class="p">(</span><span class="n">directory</span><span class="p">)</span><span class="o">.</span><span class="n">rglob</span><span class="p">(</span><span class="s2">"*.json"</span><span class="p">))</span></pre></div> 135 </div> 136 </div> 137 <div class='clearall'></div> 138 <div class='section' id='section-8'> 139 <div class='docs'> 140 <div class='octowrap'> 141 <a class='octothorpe' href='#section-8'>#</a> 142 </div> 143 <p>Iterate over all files found by previous command.</p> 144 </div> 145 <div class='code'> 146 <div class="highlight"><pre> <span class="k">for</span> <span class="nb">file</span> <span class="ow">in</span> <span class="n">files</span><span class="p">:</span> 147 <span class="k">try</span><span class="p">:</span></pre></div> 148 </div> 149 </div> 150 <div class='clearall'></div> 151 <div class='section' id='section-9'> 152 <div class='docs'> 153 <div class='octowrap'> 154 <a class='octothorpe' href='#section-9'>#</a> 155 </div> 156 <p>If the file can be opened and loaded as JSON, everything is fine.</p> 157 </div> 158 <div class='code'> 159 <div class="highlight"><pre> <span class="k">with</span> <span class="nb">file</span><span class="o">.</span><span class="n">open</span><span class="p">()</span> <span class="k">as</span> <span class="n">fin</span><span class="p">:</span></pre></div> 160 </div> 161 </div> 162 <div class='clearall'></div> 163 <div class='section' id='section-10'> 164 <div class='docs'> 165 <div class='octowrap'> 166 <a class='octothorpe' href='#section-10'>#</a> 167 </div> 168 <p>Try to load and parse the content of JSON file.</p> 169 </div> 170 <div class='code'> 171 <div class="highlight"><pre> <span class="n">obj</span> <span class="o">=</span> <span class="n">load</span><span class="p">(</span><span class="n">fin</span><span class="p">)</span></pre></div> 172 </div> 173 </div> 174 <div class='clearall'></div> 175 <div class='section' id='section-11'> 176 <div class='docs'> 177 <div class='octowrap'> 178 <a class='octothorpe' href='#section-11'>#</a> 179 </div> 180 <p>At this point the JSON has been loaded and parsed correctly.</p> 181 </div> 182 <div class='code'> 183 <div class="highlight"><pre> <span class="k">if</span> <span class="n">verbose</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span><span class="p">:</span> 184 <span class="k">print</span><span class="p">(</span><span class="s2">"{} is valid"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">file</span><span class="p">))</span> 185 186 <span class="n">passes</span> <span class="o">+=</span> <span class="mi">1</span> 187 <span class="k">except</span> <span class="ne">ValueError</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span></pre></div> 188 </div> 189 </div> 190 <div class='clearall'></div> 191 <div class='section' id='section-12'> 192 <div class='docs'> 193 <div class='octowrap'> 194 <a class='octothorpe' href='#section-12'>#</a> 195 </div> 196 <p>There are several reasons and possibilities why the file can not 197 be read as JSON, so we just print the error message taken from 198 exception object.</p> 199 </div> 200 <div class='code'> 201 <div class="highlight"><pre> <span class="k">print</span><span class="p">(</span><span class="s2">"{} is invalid"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">file</span><span class="p">))</span> 202 <span class="n">failures</span> <span class="o">+=</span> <span class="mi">1</span> 203 <span class="k">print</span><span class="p">(</span><span class="n">e</span><span class="p">)</span></pre></div> 204 </div> 205 </div> 206 <div class='clearall'></div> 207 <div class='section' id='section-13'> 208 <div class='docs'> 209 <div class='octowrap'> 210 <a class='octothorpe' href='#section-13'>#</a> 211 </div> 212 <p>Just the counters needs to be returned because all other informations 213 about problems have been displayed already.</p> 214 </div> 215 <div class='code'> 216 <div class="highlight"><pre> <span class="k">return</span> <span class="n">passes</span><span class="p">,</span> <span class="n">failures</span></pre></div> 217 </div> 218 </div> 219 <div class='clearall'></div> 220 <div class='section' id='section-14'> 221 <div class='docs'> 222 <div class='octowrap'> 223 <a class='octothorpe' href='#section-14'>#</a> 224 </div> 225 <p>Display report about number of passes and failures.</p> 226 </div> 227 <div class='code'> 228 <div class="highlight"><pre><span class="k">def</span> <span class="nf">display_report</span><span class="p">(</span><span class="n">passes</span><span class="p">,</span> <span class="n">failures</span><span class="p">,</span> <span class="n">nocolors</span><span class="p">):</span></pre></div> 229 </div> 230 </div> 231 <div class='clearall'></div> 232 <div class='section' id='section-15'> 233 <div class='docs'> 234 <div class='octowrap'> 235 <a class='octothorpe' href='#section-15'>#</a> 236 </div> 237 <p>First of all, we need to setup colors to be displayed on terminal. Colors 238 are displayed by using terminal escape control codes. When color output 239 are not enabled on command line, we can simply use empty strings in 240 output instead of real color escape codes.</p> 241 </div> 242 <div class='code'> 243 <div class="highlight"><pre> <span class="n">red_background</span> <span class="o">=</span> <span class="n">green_background</span> <span class="o">=</span> <span class="n">magenta_background</span> <span class="o">=</span> <span class="n">no_color</span> <span class="o">=</span> <span class="s2">""</span></pre></div> 244 </div> 245 </div> 246 <div class='clearall'></div> 247 <div class='section' id='section-16'> 248 <div class='docs'> 249 <div class='octowrap'> 250 <a class='octothorpe' href='#section-16'>#</a> 251 </div> 252 <p>If colors are enabled by command line parameter, use control sequence 253 returned by <code>tput</code> command.</p> 254 </div> 255 <div class='code'> 256 <div class="highlight"><pre> <span class="k">if</span> <span class="ow">not</span> <span class="n">nocolors</span><span class="p">:</span> 257 <span class="n">red_background</span> <span class="o">=</span> <span class="n">read_control_code</span><span class="p">(</span><span class="s2">"setab 1"</span><span class="p">)</span> 258 <span class="n">green_background</span> <span class="o">=</span> <span class="n">read_control_code</span><span class="p">(</span><span class="s2">"setab 2"</span><span class="p">)</span> 259 <span class="n">magenta_background</span> <span class="o">=</span> <span class="n">read_control_code</span><span class="p">(</span><span class="s2">"setab 5"</span><span class="p">)</span> 260 <span class="n">no_color</span> <span class="o">=</span> <span class="n">read_control_code</span><span class="p">(</span><span class="s2">"sgr0"</span><span class="p">)</span></pre></div> 261 </div> 262 </div> 263 <div class='clearall'></div> 264 <div class='section' id='section-17'> 265 <div class='docs'> 266 <div class='octowrap'> 267 <a class='octothorpe' href='#section-17'>#</a> 268 </div> 269 <p>There are four possible outcomes of JSON check: 270 1. no JSON files has been found 271 2. all files are ok 272 3. none of JSON files can be read and parsed 273 4. some files can be read and parsed, some can not</p> 274 </div> 275 <div class='code'> 276 <div class="highlight"><pre> <span class="k">if</span> <span class="n">failures</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span></pre></div> 277 </div> 278 </div> 279 <div class='clearall'></div> 280 <div class='section' id='section-18'> 281 <div class='docs'> 282 <div class='octowrap'> 283 <a class='octothorpe' href='#section-18'>#</a> 284 </div> 285 <p>If there are no failures, then check if any JSON file has been found at all.</p> 286 </div> 287 <div class='code'> 288 <div class="highlight"><pre> <span class="k">if</span> <span class="n">passes</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span> 289 <span class="k">print</span><span class="p">(</span><span class="s2">"{}[WARN]{}: no JSON files detected"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">magenta_background</span><span class="p">,</span> <span class="n">no_color</span><span class="p">))</span> 290 <span class="k">else</span><span class="p">:</span> 291 <span class="k">print</span><span class="p">(</span><span class="s2">"{}[OK]{}: all JSONs have proper format"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">green_background</span><span class="p">,</span> <span class="n">no_color</span><span class="p">))</span> 292 <span class="k">else</span><span class="p">:</span> 293 <span class="k">print</span><span class="p">(</span><span class="s2">"{}[FAIL]{}: invalid JSON(s) detected"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">red_background</span><span class="p">,</span> <span class="n">no_color</span><span class="p">))</span></pre></div> 294 </div> 295 </div> 296 <div class='clearall'></div> 297 <div class='section' id='section-19'> 298 <div class='docs'> 299 <div class='octowrap'> 300 <a class='octothorpe' href='#section-19'>#</a> 301 </div> 302 <p>Print just number of passes and failures at the end, as this information 303 can be processed on CI.</p> 304 </div> 305 <div class='code'> 306 <div class="highlight"><pre> <span class="k">print</span><span class="p">(</span><span class="s2">"{} passes"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">passes</span><span class="p">))</span> 307 <span class="k">print</span><span class="p">(</span><span class="s2">"{} failures"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">failures</span><span class="p">))</span></pre></div> 308 </div> 309 </div> 310 <div class='clearall'></div> 311 <div class='section' id='section-20'> 312 <div class='docs'> 313 <div class='octowrap'> 314 <a class='octothorpe' href='#section-20'>#</a> 315 </div> 316 <p>Entry point to this tool.</p> 317 </div> 318 <div class='code'> 319 <div class="highlight"><pre><span class="k">def</span> <span class="nf">main</span><span class="p">():</span></pre></div> 320 </div> 321 </div> 322 <div class='clearall'></div> 323 <div class='section' id='section-21'> 324 <div class='docs'> 325 <div class='octowrap'> 326 <a class='octothorpe' href='#section-21'>#</a> 327 </div> 328 <p>First of all, we need to specify all command line flags that are 329 recognized by this tool.</p> 330 </div> 331 <div class='code'> 332 <div class="highlight"><pre> <span class="n">parser</span> <span class="o">=</span> <span class="n">ArgumentParser</span><span class="p">()</span> 333 <span class="n">parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span><span class="s2">"-v"</span><span class="p">,</span> <span class="s2">"--verbose"</span><span class="p">,</span> <span class="n">dest</span><span class="o">=</span><span class="s2">"verbose"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"make it verbose"</span><span class="p">,</span> 334 <span class="n">action</span><span class="o">=</span><span class="s2">"store_true"</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="bp">None</span><span class="p">)</span> 335 <span class="n">parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span><span class="s2">"-n"</span><span class="p">,</span> <span class="s2">"--no-colors"</span><span class="p">,</span> <span class="n">dest</span><span class="o">=</span><span class="s2">"nocolors"</span><span class="p">,</span> <span class="n">help</span><span class="o">=</span><span class="s2">"disable color output"</span><span class="p">,</span> 336 <span class="n">action</span><span class="o">=</span><span class="s2">"store_true"</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="bp">None</span><span class="p">)</span> 337 <span class="n">parser</span><span class="o">.</span><span class="n">add_argument</span><span class="p">(</span><span class="s2">"-d"</span><span class="p">,</span> <span class="s2">"--directory"</span><span class="p">,</span> <span class="n">dest</span><span class="o">=</span><span class="s2">"directory"</span><span class="p">,</span> 338 <span class="n">help</span><span class="o">=</span><span class="s2">"directory with JSON files to check"</span><span class="p">,</span> 339 <span class="n">action</span><span class="o">=</span><span class="s2">"store"</span><span class="p">,</span> <span class="n">default</span><span class="o">=</span><span class="s2">"."</span><span class="p">)</span></pre></div> 340 </div> 341 </div> 342 <div class='clearall'></div> 343 <div class='section' id='section-22'> 344 <div class='docs'> 345 <div class='octowrap'> 346 <a class='octothorpe' href='#section-22'>#</a> 347 </div> 348 <p>Now it is time to parse flags, check the actual content of command line 349 and fill in the object stored in variable named <code>args</code>.</p> 350 </div> 351 <div class='code'> 352 <div class="highlight"><pre> <span class="n">args</span> <span class="o">=</span> <span class="n">parser</span><span class="o">.</span><span class="n">parse_args</span><span class="p">()</span></pre></div> 353 </div> 354 </div> 355 <div class='clearall'></div> 356 <div class='section' id='section-23'> 357 <div class='docs'> 358 <div class='octowrap'> 359 <a class='octothorpe' href='#section-23'>#</a> 360 </div> 361 <p>Check all JSON files, display problems, and get counters with number of 362 passes and failures.</p> 363 </div> 364 <div class='code'> 365 <div class="highlight"><pre> <span class="n">passes</span><span class="p">,</span> <span class="n">failures</span> <span class="o">=</span> <span class="n">check_jsons</span><span class="p">(</span><span class="n">args</span><span class="o">.</span><span class="n">verbose</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">directory</span><span class="p">)</span></pre></div> 366 </div> 367 </div> 368 <div class='clearall'></div> 369 <div class='section' id='section-24'> 370 <div class='docs'> 371 <div class='octowrap'> 372 <a class='octothorpe' href='#section-24'>#</a> 373 </div> 374 <p>Display detailed report and summary as well.</p> 375 </div> 376 <div class='code'> 377 <div class="highlight"><pre> <span class="n">display_report</span><span class="p">(</span><span class="n">passes</span><span class="p">,</span> <span class="n">failures</span><span class="p">,</span> <span class="n">args</span><span class="o">.</span><span class="n">nocolors</span><span class="p">)</span></pre></div> 378 </div> 379 </div> 380 <div class='clearall'></div> 381 <div class='section' id='section-25'> 382 <div class='docs'> 383 <div class='octowrap'> 384 <a class='octothorpe' href='#section-25'>#</a> 385 </div> 386 <p>If any error is found, return with exit code check to non-zero value.</p> 387 </div> 388 <div class='code'> 389 <div class="highlight"><pre> <span class="k">if</span> <span class="n">failures</span> <span class="o">></span> <span class="mi">0</span><span class="p">:</span> 390 <span class="nb">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span></pre></div> 391 </div> 392 </div> 393 <div class='clearall'></div> 394 <div class='section' id='section-26'> 395 <div class='docs'> 396 <div class='octowrap'> 397 <a class='octothorpe' href='#section-26'>#</a> 398 </div> 399 <p>If this script is started from command line, run the <code>main</code> function 400 which represents entry point to the processing. 401 Entry point to this tool.</p> 402 </div> 403 <div class='code'> 404 <div class="highlight"><pre><span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span></pre></div> 405 </div> 406 </div> 407 <div class='clearall'></div> 408 <div class='section' id='section-27'> 409 <div class='docs'> 410 <div class='octowrap'> 411 <a class='octothorpe' href='#section-27'>#</a> 412 </div> 413 414 </div> 415 <div class='code'> 416 <div class="highlight"><pre> <span class="n">main</span><span class="p">()</span> 417 418 </pre></div> 419 </div> 420 </div> 421 <div class='clearall'></div> 422 </div> 423 </body>