Add CLCACHE_PROFILE environment variable to enable profiling

[siu]

Not sure if you want to merge this in the main branch but it is useful to have this option to profile clcache in production environments.

[codecov-io]

Current coverage is 78.34% (diff: 40.00%)

Merging #207 into master will decrease coverage by 0.22%

@@             master       #207   diff @@
==========================================
  Files             1          1          
  Lines           929        933     +4   
  Methods           0          0          
  Messages          0          0          
  Branches        155        156     +1   
==========================================
+ Hits            730        731     +1   
- Misses          166        168     +2   
- Partials         33         34     +1   

Powered by Codecov. Last update e25b61a...04c5c78

[frerich]

I think this is really interesting! Over the last couple of weeks, I was contemplating of adding exactly such a switch to clcache myself, but I was not aware that there's a ready-made Python module for this.

Two things:

1. Can you post some sample output of this module?
2. Can you add documentation of this new variable to the README file?

[hubx]

I think for representative workloads we want to run something like cProfile.run('main()', filename=HashAlgorithm(','.join(sys.argv).encode()).hexdigest()) and aggregate the generated pstats file later.

[frerich]

@hubx I very much agree, some sort of aggregation would be good. It appears this is not too hard to do with the pstats module.

@siu Can you maybe have a look at #211 and see how it works for you? It's basically an extension of your idea, it also introduces a CLCACHE_PROFILE environment variable, but writes the generated data to external files. The generated files can then be processed into a final compound report using a little showprofilereport.py script.

[siu]

Without testing it (working on linux today) #211 satisfies our needs and more, this PR can be closed.

[frerich]

Closing this in favor of #211 .