51ae6f71c381e55e1a35100f92201351d2a5f34e
[ghc.git] / docs / users_guide / profiling.xml
1 <?xml version="1.0" encoding="iso-8859-1"?>
2 <chapter id="profiling">
3 <title>Profiling</title>
4 <indexterm><primary>profiling</primary>
5 </indexterm>
6 <indexterm><primary>cost-centre profiling</primary></indexterm>
7
8 <para> Glasgow Haskell comes with a time and space profiling
9 system. Its purpose is to help you improve your understanding of
10 your program's execution behaviour, so you can improve it.</para>
11
12 <para> Any comments, suggestions and/or improvements you have are
13 welcome. Recommended &ldquo;profiling tricks&rdquo; would be
14 especially cool! </para>
15
16 <para>Profiling a program is a three-step process:</para>
17
18 <orderedlist>
19 <listitem>
20 <para> Re-compile your program for profiling with the
21 <literal>-prof</literal> option, and probably one of the
22 <literal>-auto</literal> or <literal>-auto-all</literal>
23 options. These options are described in more detail in <xref
24 linkend="prof-compiler-options"/> </para>
25 <indexterm><primary><literal>-prof</literal></primary>
26 </indexterm>
27 <indexterm><primary><literal>-auto</literal></primary>
28 </indexterm>
29 <indexterm><primary><literal>-auto-all</literal></primary>
30 </indexterm>
31 </listitem>
32
33 <listitem>
34 <para> Run your program with one of the profiling options, eg.
35 <literal>+RTS -p -RTS</literal>. This generates a file of
36 profiling information. Note that multi-processor execution
37 (e.g. <literal>+RTS -N2</literal>) is not supported while
38 profiling.</para>
39 <indexterm><primary><option>-p</option></primary><secondary>RTS
40 option</secondary></indexterm>
41 </listitem>
42
43 <listitem>
44 <para> Examine the generated profiling information, using one of
45 GHC's profiling tools. The tool to use will depend on the kind
46 of profiling information generated.</para>
47 </listitem>
48
49 </orderedlist>
50
51 <sect1 id="cost-centres">
52 <title>Cost centres and cost-centre stacks</title>
53
54 <para>GHC's profiling system assigns <firstterm>costs</firstterm>
55 to <firstterm>cost centres</firstterm>. A cost is simply the time
56 or space required to evaluate an expression. Cost centres are
57 program annotations around expressions; all costs incurred by the
58 annotated expression are assigned to the enclosing cost centre.
59 Furthermore, GHC will remember the stack of enclosing cost centres
60 for any given expression at run-time and generate a call-graph of
61 cost attributions.</para>
62
63 <para>Let's take a look at an example:</para>
64
65 <programlisting>
66 main = print (nfib 25)
67 nfib n = if n &lt; 2 then 1 else nfib (n-1) + nfib (n-2)
68 </programlisting>
69
70 <para>Compile and run this program as follows:</para>
71
72 <screen>
73 $ ghc -prof -auto-all -o Main Main.hs
74 $ ./Main +RTS -p
75 121393
76 $
77 </screen>
78
79 <para>When a GHC-compiled program is run with the
80 <option>-p</option> RTS option, it generates a file called
81 <filename>&lt;prog&gt;.prof</filename>. In this case, the file
82 will contain something like this:</para>
83
84 <screen>
85 Fri May 12 14:06 2000 Time and Allocation Profiling Report (Final)
86
87 Main +RTS -p -RTS
88
89 total time = 0.14 secs (7 ticks @ 20 ms)
90 total alloc = 8,741,204 bytes (excludes profiling overheads)
91
92 COST CENTRE MODULE %time %alloc
93
94 nfib Main 100.0 100.0
95
96
97 individual inherited
98 COST CENTRE MODULE entries %time %alloc %time %alloc
99
100 MAIN MAIN 0 0.0 0.0 100.0 100.0
101 main Main 0 0.0 0.0 0.0 0.0
102 CAF PrelHandle 3 0.0 0.0 0.0 0.0
103 CAF PrelAddr 1 0.0 0.0 0.0 0.0
104 CAF Main 6 0.0 0.0 100.0 100.0
105 main Main 1 0.0 0.0 100.0 100.0
106 nfib Main 242785 100.0 100.0 100.0 100.0
107 </screen>
108
109
110 <para>The first part of the file gives the program name and
111 options, and the total time and total memory allocation measured
112 during the run of the program (note that the total memory
113 allocation figure isn't the same as the amount of
114 <emphasis>live</emphasis> memory needed by the program at any one
115 time; the latter can be determined using heap profiling, which we
116 will describe shortly).</para>
117
118 <para>The second part of the file is a break-down by cost centre
119 of the most costly functions in the program. In this case, there
120 was only one significant function in the program, namely
121 <function>nfib</function>, and it was responsible for 100&percnt;
122 of both the time and allocation costs of the program.</para>
123
124 <para>The third and final section of the file gives a profile
125 break-down by cost-centre stack. This is roughly a call-graph
126 profile of the program. In the example above, it is clear that
127 the costly call to <function>nfib</function> came from
128 <function>main</function>.</para>
129
130 <para>The time and allocation incurred by a given part of the
131 program is displayed in two ways: &ldquo;individual&rdquo;, which
132 are the costs incurred by the code covered by this cost centre
133 stack alone, and &ldquo;inherited&rdquo;, which includes the costs
134 incurred by all the children of this node.</para>
135
136 <para>The usefulness of cost-centre stacks is better demonstrated
137 by modifying the example slightly:</para>
138
139 <programlisting>
140 main = print (f 25 + g 25)
141 f n = nfib n
142 g n = nfib (n `div` 2)
143 nfib n = if n &lt; 2 then 1 else nfib (n-1) + nfib (n-2)
144 </programlisting>
145
146 <para>Compile and run this program as before, and take a look at
147 the new profiling results:</para>
148
149 <screen>
150 COST CENTRE MODULE scc %time %alloc %time %alloc
151
152 MAIN MAIN 0 0.0 0.0 100.0 100.0
153 main Main 0 0.0 0.0 0.0 0.0
154 CAF PrelHandle 3 0.0 0.0 0.0 0.0
155 CAF PrelAddr 1 0.0 0.0 0.0 0.0
156 CAF Main 9 0.0 0.0 100.0 100.0
157 main Main 1 0.0 0.0 100.0 100.0
158 g Main 1 0.0 0.0 0.0 0.2
159 nfib Main 465 0.0 0.2 0.0 0.2
160 f Main 1 0.0 0.0 100.0 99.8
161 nfib Main 242785 100.0 99.8 100.0 99.8
162 </screen>
163
164 <para>Now although we had two calls to <function>nfib</function>
165 in the program, it is immediately clear that it was the call from
166 <function>f</function> which took all the time.</para>
167
168 <para>The actual meaning of the various columns in the output is:</para>
169
170 <variablelist>
171 <varlistentry>
172 <term>entries</term>
173 <listitem>
174 <para>The number of times this particular point in the call
175 graph was entered.</para>
176 </listitem>
177 </varlistentry>
178
179 <varlistentry>
180 <term>individual &percnt;time</term>
181 <listitem>
182 <para>The percentage of the total run time of the program
183 spent at this point in the call graph.</para>
184 </listitem>
185 </varlistentry>
186
187 <varlistentry>
188 <term>individual &percnt;alloc</term>
189 <listitem>
190 <para>The percentage of the total memory allocations
191 (excluding profiling overheads) of the program made by this
192 call.</para>
193 </listitem>
194 </varlistentry>
195
196 <varlistentry>
197 <term>inherited &percnt;time</term>
198 <listitem>
199 <para>The percentage of the total run time of the program
200 spent below this point in the call graph.</para>
201 </listitem>
202 </varlistentry>
203
204 <varlistentry>
205 <term>inherited &percnt;alloc</term>
206 <listitem>
207 <para>The percentage of the total memory allocations
208 (excluding profiling overheads) of the program made by this
209 call and all of its sub-calls.</para>
210 </listitem>
211 </varlistentry>
212 </variablelist>
213
214 <para>In addition you can use the <option>-P</option> RTS option
215 <indexterm><primary><option>-P</option></primary></indexterm> to
216 get the following additional information:</para>
217
218 <variablelist>
219 <varlistentry>
220 <term><literal>ticks</literal></term>
221 <listitem>
222 <para>The raw number of time &ldquo;ticks&rdquo; which were
223 attributed to this cost-centre; from this, we get the
224 <literal>&percnt;time</literal> figure mentioned
225 above.</para>
226 </listitem>
227 </varlistentry>
228
229 <varlistentry>
230 <term><literal>bytes</literal></term>
231 <listitem>
232 <para>Number of bytes allocated in the heap while in this
233 cost-centre; again, this is the raw number from which we get
234 the <literal>&percnt;alloc</literal> figure mentioned
235 above.</para>
236 </listitem>
237 </varlistentry>
238 </variablelist>
239
240 <para>What about recursive functions, and mutually recursive
241 groups of functions? Where are the costs attributed? Well,
242 although GHC does keep information about which groups of functions
243 called each other recursively, this information isn't displayed in
244 the basic time and allocation profile, instead the call-graph is
245 flattened into a tree.</para>
246
247 <sect2><title>Inserting cost centres by hand</title>
248
249 <para>Cost centres are just program annotations. When you say
250 <option>-auto-all</option> to the compiler, it automatically
251 inserts a cost centre annotation around every top-level function
252 in your program, but you are entirely free to add the cost
253 centre annotations yourself.</para>
254
255 <para>The syntax of a cost centre annotation is</para>
256
257 <programlisting>
258 {-# SCC "name" #-} &lt;expression&gt;
259 </programlisting>
260
261 <para>where <literal>"name"</literal> is an arbitrary string,
262 that will become the name of your cost centre as it appears
263 in the profiling output, and
264 <literal>&lt;expression&gt;</literal> is any Haskell
265 expression. An <literal>SCC</literal> annotation extends as
266 far to the right as possible when parsing. (SCC stands for "Set
267 Cost Centre").</para>
268
269 </sect2>
270
271 <sect2 id="prof-rules">
272 <title>Rules for attributing costs</title>
273
274 <para>The cost of evaluating any expression in your program is
275 attributed to a cost-centre stack using the following rules:</para>
276
277 <itemizedlist>
278 <listitem>
279 <para>If the expression is part of the
280 <firstterm>one-off</firstterm> costs of evaluating the
281 enclosing top-level definition, then costs are attributed to
282 the stack of lexically enclosing <literal>SCC</literal>
283 annotations on top of the special <literal>CAF</literal>
284 cost-centre. </para>
285 </listitem>
286
287 <listitem>
288 <para>Otherwise, costs are attributed to the stack of
289 lexically-enclosing <literal>SCC</literal> annotations,
290 appended to the cost-centre stack in effect at the
291 <firstterm>call site</firstterm> of the current top-level
292 definition<footnote> <para>The call-site is just the place
293 in the source code which mentions the particular function or
294 variable.</para></footnote>. Notice that this is a recursive
295 definition.</para>
296 </listitem>
297
298 <listitem>
299 <para>Time spent in foreign code (see <xref linkend="ffi"/>)
300 is always attributed to the cost centre in force at the
301 Haskell call-site of the foreign function.</para>
302 </listitem>
303 </itemizedlist>
304
305 <para>What do we mean by one-off costs? Well, Haskell is a lazy
306 language, and certain expressions are only ever evaluated once.
307 For example, if we write:</para>
308
309 <programlisting>
310 x = nfib 25
311 </programlisting>
312
313 <para>then <varname>x</varname> will only be evaluated once (if
314 at all), and subsequent demands for <varname>x</varname> will
315 immediately get to see the cached result. The definition
316 <varname>x</varname> is called a CAF (Constant Applicative
317 Form), because it has no arguments.</para>
318
319 <para>For the purposes of profiling, we say that the expression
320 <literal>nfib 25</literal> belongs to the one-off costs of
321 evaluating <varname>x</varname>.</para>
322
323 <para>Since one-off costs aren't strictly speaking part of the
324 call-graph of the program, they are attributed to a special
325 top-level cost centre, <literal>CAF</literal>. There may be one
326 <literal>CAF</literal> cost centre for each module (the
327 default), or one for each top-level definition with any one-off
328 costs (this behaviour can be selected by giving GHC the
329 <option>-caf-all</option> flag).</para>
330
331 <indexterm><primary><literal>-caf-all</literal></primary>
332 </indexterm>
333
334 <para>If you think you have a weird profile, or the call-graph
335 doesn't look like you expect it to, feel free to send it (and
336 your program) to us at
337 <email>glasgow-haskell-bugs@haskell.org</email>.</para>
338 </sect2>
339 </sect1>
340
341 <sect1 id="prof-compiler-options">
342 <title>Compiler options for profiling</title>
343
344 <indexterm><primary>profiling</primary><secondary>options</secondary></indexterm>
345 <indexterm><primary>options</primary><secondary>for profiling</secondary></indexterm>
346
347 <variablelist>
348 <varlistentry>
349 <term>
350 <option>-prof</option>:
351 <indexterm><primary><option>-prof</option></primary></indexterm>
352 </term>
353 <listitem>
354 <para> To make use of the profiling system
355 <emphasis>all</emphasis> modules must be compiled and linked
356 with the <option>-prof</option> option. Any
357 <literal>SCC</literal> annotations you've put in your source
358 will spring to life.</para>
359
360 <para> Without a <option>-prof</option> option, your
361 <literal>SCC</literal>s are ignored; so you can compile
362 <literal>SCC</literal>-laden code without changing
363 it.</para>
364 </listitem>
365 </varlistentry>
366 </variablelist>
367
368 <para>There are a few other profiling-related compilation options.
369 Use them <emphasis>in addition to</emphasis>
370 <option>-prof</option>. These do not have to be used consistently
371 for all modules in a program.</para>
372
373 <variablelist>
374 <varlistentry>
375 <term>
376 <option>-auto</option>:
377 <indexterm><primary><option>-auto</option></primary></indexterm>
378 <indexterm><primary>cost centres</primary><secondary>automatically inserting</secondary></indexterm>
379 </term>
380 <listitem>
381 <para> GHC will automatically add
382 <function>&lowbar;scc&lowbar;</function> constructs for all
383 top-level, exported functions.</para>
384 </listitem>
385 </varlistentry>
386
387 <varlistentry>
388 <term>
389 <option>-auto-all</option>:
390 <indexterm><primary><option>-auto-all</option></primary></indexterm>
391 </term>
392 <listitem>
393 <para> <emphasis>All</emphasis> top-level functions,
394 exported or not, will be automatically
395 <function>&lowbar;scc&lowbar;</function>'d.</para>
396 </listitem>
397 </varlistentry>
398
399 <varlistentry>
400 <term>
401 <option>-caf-all</option>:
402 <indexterm><primary><option>-caf-all</option></primary></indexterm>
403 </term>
404 <listitem>
405 <para> The costs of all CAFs in a module are usually
406 attributed to one &ldquo;big&rdquo; CAF cost-centre. With
407 this option, all CAFs get their own cost-centre. An
408 &ldquo;if all else fails&rdquo; option&hellip;</para>
409 </listitem>
410 </varlistentry>
411
412 <varlistentry>
413 <term>
414 <option>-ignore-scc</option>:
415 <indexterm><primary><option>-ignore-scc</option></primary></indexterm>
416 </term>
417 <listitem>
418 <para>Ignore any <function>&lowbar;scc&lowbar;</function>
419 constructs, so a module which already has
420 <function>&lowbar;scc&lowbar;</function>s can be compiled
421 for profiling with the annotations ignored.</para>
422 </listitem>
423 </varlistentry>
424
425 </variablelist>
426
427 </sect1>
428
429 <sect1 id="prof-time-options">
430 <title>Time and allocation profiling</title>
431
432 <para>To generate a time and allocation profile, give one of the
433 following RTS options to the compiled program when you run it (RTS
434 options should be enclosed between <literal>+RTS...-RTS</literal>
435 as usual):</para>
436
437 <variablelist>
438 <varlistentry>
439 <term>
440 <option>-p</option> or <option>-P</option>:
441 <indexterm><primary><option>-p</option></primary></indexterm>
442 <indexterm><primary><option>-P</option></primary></indexterm>
443 <indexterm><primary>time profile</primary></indexterm>
444 </term>
445 <listitem>
446 <para>The <option>-p</option> option produces a standard
447 <emphasis>time profile</emphasis> report. It is written
448 into the file
449 <filename><replaceable>program</replaceable>.prof</filename>.</para>
450
451 <para>The <option>-P</option> option produces a more
452 detailed report containing the actual time and allocation
453 data as well. (Not used much.)</para>
454 </listitem>
455 </varlistentry>
456
457 <varlistentry>
458 <term>
459 <option>-xc</option>
460 <indexterm><primary><option>-xc</option></primary><secondary>RTS option</secondary></indexterm>
461 </term>
462 <listitem>
463 <para>This option makes use of the extra information
464 maintained by the cost-centre-stack profiler to provide
465 useful information about the location of runtime errors.
466 See <xref linkend="rts-options-debugging"/>.</para>
467 </listitem>
468 </varlistentry>
469
470 </variablelist>
471
472 </sect1>
473
474 <sect1 id="prof-heap">
475 <title>Profiling memory usage</title>
476
477 <para>In addition to profiling the time and allocation behaviour
478 of your program, you can also generate a graph of its memory usage
479 over time. This is useful for detecting the causes of
480 <firstterm>space leaks</firstterm>, when your program holds on to
481 more memory at run-time that it needs to. Space leaks lead to
482 longer run-times due to heavy garbage collector activity, and may
483 even cause the program to run out of memory altogether.</para>
484
485 <para>To generate a heap profile from your program:</para>
486
487 <orderedlist>
488 <listitem>
489 <para>Compile the program for profiling (<xref
490 linkend="prof-compiler-options"/>).</para>
491 </listitem>
492 <listitem>
493 <para>Run it with one of the heap profiling options described
494 below (eg. <option>-hc</option> for a basic producer profile).
495 This generates the file
496 <filename><replaceable>prog</replaceable>.hp</filename>.</para>
497 </listitem>
498 <listitem>
499 <para>Run <command>hp2ps</command> to produce a Postscript
500 file,
501 <filename><replaceable>prog</replaceable>.ps</filename>. The
502 <command>hp2ps</command> utility is described in detail in
503 <xref linkend="hp2ps"/>.</para>
504 </listitem>
505 <listitem>
506 <para>Display the heap profile using a postscript viewer such
507 as <application>Ghostview</application>, or print it out on a
508 Postscript-capable printer.</para>
509 </listitem>
510 </orderedlist>
511
512 <sect2 id="rts-options-heap-prof">
513 <title>RTS options for heap profiling</title>
514
515 <para>There are several different kinds of heap profile that can
516 be generated. All the different profile types yield a graph of
517 live heap against time, but they differ in how the live heap is
518 broken down into bands. The following RTS options select which
519 break-down to use:</para>
520
521 <variablelist>
522 <varlistentry>
523 <term>
524 <option>-hc</option>
525 <indexterm><primary><option>-hc</option></primary><secondary>RTS option</secondary></indexterm>
526 </term>
527 <listitem>
528 <para>Breaks down the graph by the cost-centre stack which
529 produced the data.</para>
530 </listitem>
531 </varlistentry>
532
533 <varlistentry>
534 <term>
535 <option>-hm</option>
536 <indexterm><primary><option>-hm</option></primary><secondary>RTS option</secondary></indexterm>
537 </term>
538 <listitem>
539 <para>Break down the live heap by the module containing
540 the code which produced the data.</para>
541 </listitem>
542 </varlistentry>
543
544 <varlistentry>
545 <term>
546 <option>-hd</option>
547 <indexterm><primary><option>-hd</option></primary><secondary>RTS option</secondary></indexterm>
548 </term>
549 <listitem>
550 <para>Breaks down the graph by <firstterm>closure
551 description</firstterm>. For actual data, the description
552 is just the constructor name, for other closures it is a
553 compiler-generated string identifying the closure.</para>
554 </listitem>
555 </varlistentry>
556
557 <varlistentry>
558 <term>
559 <option>-hy</option>
560 <indexterm><primary><option>-hy</option></primary><secondary>RTS option</secondary></indexterm>
561 </term>
562 <listitem>
563 <para>Breaks down the graph by
564 <firstterm>type</firstterm>. For closures which have
565 function type or unknown/polymorphic type, the string will
566 represent an approximation to the actual type.</para>
567 </listitem>
568 </varlistentry>
569
570 <varlistentry>
571 <term>
572 <option>-hr</option>
573 <indexterm><primary><option>-hr</option></primary><secondary>RTS option</secondary></indexterm>
574 </term>
575 <listitem>
576 <para>Break down the graph by <firstterm>retainer
577 set</firstterm>. Retainer profiling is described in more
578 detail below (<xref linkend="retainer-prof"/>).</para>
579 </listitem>
580 </varlistentry>
581
582 <varlistentry>
583 <term>
584 <option>-hb</option>
585 <indexterm><primary><option>-hb</option></primary><secondary>RTS option</secondary></indexterm>
586 </term>
587 <listitem>
588 <para>Break down the graph by
589 <firstterm>biography</firstterm>. Biographical profiling
590 is described in more detail below (<xref
591 linkend="biography-prof"/>).</para>
592 </listitem>
593 </varlistentry>
594 </variablelist>
595
596 <para>In addition, the profile can be restricted to heap data
597 which satisfies certain criteria - for example, you might want
598 to display a profile by type but only for data produced by a
599 certain module, or a profile by retainer for a certain type of
600 data. Restrictions are specified as follows:</para>
601
602 <variablelist>
603 <varlistentry>
604 <term>
605 <option>-hc</option><replaceable>name</replaceable>,...
606 <indexterm><primary><option>-hc</option></primary><secondary>RTS option</secondary></indexterm>
607 </term>
608 <listitem>
609 <para>Restrict the profile to closures produced by
610 cost-centre stacks with one of the specified cost centres
611 at the top.</para>
612 </listitem>
613 </varlistentry>
614
615 <varlistentry>
616 <term>
617 <option>-hC</option><replaceable>name</replaceable>,...
618 <indexterm><primary><option>-hC</option></primary><secondary>RTS option</secondary></indexterm>
619 </term>
620 <listitem>
621 <para>Restrict the profile to closures produced by
622 cost-centre stacks with one of the specified cost centres
623 anywhere in the stack.</para>
624 </listitem>
625 </varlistentry>
626
627 <varlistentry>
628 <term>
629 <option>-hm</option><replaceable>module</replaceable>,...
630 <indexterm><primary><option>-hm</option></primary><secondary>RTS option</secondary></indexterm>
631 </term>
632 <listitem>
633 <para>Restrict the profile to closures produced by the
634 specified modules.</para>
635 </listitem>
636 </varlistentry>
637
638 <varlistentry>
639 <term>
640 <option>-hd</option><replaceable>desc</replaceable>,...
641 <indexterm><primary><option>-hd</option></primary><secondary>RTS option</secondary></indexterm>
642 </term>
643 <listitem>
644 <para>Restrict the profile to closures with the specified
645 description strings.</para>
646 </listitem>
647 </varlistentry>
648
649 <varlistentry>
650 <term>
651 <option>-hy</option><replaceable>type</replaceable>,...
652 <indexterm><primary><option>-hy</option></primary><secondary>RTS option</secondary></indexterm>
653 </term>
654 <listitem>
655 <para>Restrict the profile to closures with the specified
656 types.</para>
657 </listitem>
658 </varlistentry>
659
660 <varlistentry>
661 <term>
662 <option>-hr</option><replaceable>cc</replaceable>,...
663 <indexterm><primary><option>-hr</option></primary><secondary>RTS option</secondary></indexterm>
664 </term>
665 <listitem>
666 <para>Restrict the profile to closures with retainer sets
667 containing cost-centre stacks with one of the specified
668 cost centres at the top.</para>
669 </listitem>
670 </varlistentry>
671
672 <varlistentry>
673 <term>
674 <option>-hb</option><replaceable>bio</replaceable>,...
675 <indexterm><primary><option>-hb</option></primary><secondary>RTS option</secondary></indexterm>
676 </term>
677 <listitem>
678 <para>Restrict the profile to closures with one of the
679 specified biographies, where
680 <replaceable>bio</replaceable> is one of
681 <literal>lag</literal>, <literal>drag</literal>,
682 <literal>void</literal>, or <literal>use</literal>.</para>
683 </listitem>
684 </varlistentry>
685 </variablelist>
686
687 <para>For example, the following options will generate a
688 retainer profile restricted to <literal>Branch</literal> and
689 <literal>Leaf</literal> constructors:</para>
690
691 <screen>
692 <replaceable>prog</replaceable> +RTS -hr -hdBranch,Leaf
693 </screen>
694
695 <para>There can only be one "break-down" option
696 (eg. <option>-hr</option> in the example above), but there is no
697 limit on the number of further restrictions that may be applied.
698 All the options may be combined, with one exception: GHC doesn't
699 currently support mixing the <option>-hr</option> and
700 <option>-hb</option> options.</para>
701
702 <para>There are three more options which relate to heap
703 profiling:</para>
704
705 <variablelist>
706 <varlistentry>
707 <term>
708 <option>-i<replaceable>secs</replaceable></option>:
709 <indexterm><primary><option>-i</option></primary></indexterm>
710 </term>
711 <listitem>
712 <para>Set the profiling (sampling) interval to
713 <replaceable>secs</replaceable> seconds (the default is
714 0.1&nbsp;second). Fractions are allowed: for example
715 <option>-i0.2</option> will get 5 samples per second.
716 This only affects heap profiling; time profiles are always
717 sampled on a 1/50 second frequency.</para>
718 </listitem>
719 </varlistentry>
720
721 <varlistentry>
722 <term>
723 <option>-xt</option>
724 <indexterm><primary><option>-xt</option></primary><secondary>RTS option</secondary></indexterm>
725 </term>
726 <listitem>
727 <para>Include the memory occupied by threads in a heap
728 profile. Each thread takes up a small area for its thread
729 state in addition to the space allocated for its stack
730 (stacks normally start small and then grow as
731 necessary).</para>
732
733 <para>This includes the main thread, so using
734 <option>-xt</option> is a good way to see how much stack
735 space the program is using.</para>
736
737 <para>Memory occupied by threads and their stacks is
738 labelled as &ldquo;TSO&rdquo; when displaying the profile
739 by closure description or type description.</para>
740 </listitem>
741 </varlistentry>
742
743 <varlistentry>
744 <term>
745 <option>-L<replaceable>num</replaceable></option>
746 <indexterm><primary><option>-L</option></primary><secondary>RTS option</secondary></indexterm>
747 </term>
748 <listitem>
749 <para>
750 Sets the maximum length of a cost-centre stack name in a
751 heap profile. Defaults to 25.
752 </para>
753 </listitem>
754 </varlistentry>
755 </variablelist>
756
757 </sect2>
758
759 <sect2 id="retainer-prof">
760 <title>Retainer Profiling</title>
761
762 <para>Retainer profiling is designed to help answer questions
763 like <quote>why is this data being retained?</quote>. We start
764 by defining what we mean by a retainer:</para>
765
766 <blockquote>
767 <para>A retainer is either the system stack, or an unevaluated
768 closure (thunk).</para>
769 </blockquote>
770
771 <para>In particular, constructors are <emphasis>not</emphasis>
772 retainers.</para>
773
774 <para>An object B retains object A if (i) B is a retainer object and
775 (ii) object A can be reached by recursively following pointers
776 starting from object B, but not meeting any other retainer
777 objects on the way. Each live object is retained by one or more
778 retainer objects, collectively called its retainer set, or its
779 <firstterm>retainer set</firstterm>, or its
780 <firstterm>retainers</firstterm>.</para>
781
782 <para>When retainer profiling is requested by giving the program
783 the <option>-hr</option> option, a graph is generated which is
784 broken down by retainer set. A retainer set is displayed as a
785 set of cost-centre stacks; because this is usually too large to
786 fit on the profile graph, each retainer set is numbered and
787 shown abbreviated on the graph along with its number, and the
788 full list of retainer sets is dumped into the file
789 <filename><replaceable>prog</replaceable>.prof</filename>.</para>
790
791 <para>Retainer profiling requires multiple passes over the live
792 heap in order to discover the full retainer set for each
793 object, which can be quite slow. So we set a limit on the
794 maximum size of a retainer set, where all retainer sets larger
795 than the maximum retainer set size are replaced by the special
796 set <literal>MANY</literal>. The maximum set size defaults to 8
797 and can be altered with the <option>-R</option> RTS
798 option:</para>
799
800 <variablelist>
801 <varlistentry>
802 <term><option>-R</option><replaceable>size</replaceable></term>
803 <listitem>
804 <para>Restrict the number of elements in a retainer set to
805 <replaceable>size</replaceable> (default 8).</para>
806 </listitem>
807 </varlistentry>
808 </variablelist>
809
810 <sect3>
811 <title>Hints for using retainer profiling</title>
812
813 <para>The definition of retainers is designed to reflect a
814 common cause of space leaks: a large structure is retained by
815 an unevaluated computation, and will be released once the
816 computation is forced. A good example is looking up a value in
817 a finite map, where unless the lookup is forced in a timely
818 manner the unevaluated lookup will cause the whole mapping to
819 be retained. These kind of space leaks can often be
820 eliminated by forcing the relevant computations to be
821 performed eagerly, using <literal>seq</literal> or strictness
822 annotations on data constructor fields.</para>
823
824 <para>Often a particular data structure is being retained by a
825 chain of unevaluated closures, only the nearest of which will
826 be reported by retainer profiling - for example A retains B, B
827 retains C, and C retains a large structure. There might be a
828 large number of Bs but only a single A, so A is really the one
829 we're interested in eliminating. However, retainer profiling
830 will in this case report B as the retainer of the large
831 structure. To move further up the chain of retainers, we can
832 ask for another retainer profile but this time restrict the
833 profile to B objects, so we get a profile of the retainers of
834 B:</para>
835
836 <screen>
837 <replaceable>prog</replaceable> +RTS -hr -hcB
838 </screen>
839
840 <para>This trick isn't foolproof, because there might be other
841 B closures in the heap which aren't the retainers we are
842 interested in, but we've found this to be a useful technique
843 in most cases.</para>
844 </sect3>
845 </sect2>
846
847 <sect2 id="biography-prof">
848 <title>Biographical Profiling</title>
849
850 <para>A typical heap object may be in one of the following four
851 states at each point in its lifetime:</para>
852
853 <itemizedlist>
854 <listitem>
855 <para>The <firstterm>lag</firstterm> stage, which is the
856 time between creation and the first use of the
857 object,</para>
858 </listitem>
859 <listitem>
860 <para>the <firstterm>use</firstterm> stage, which lasts from
861 the first use until the last use of the object, and</para>
862 </listitem>
863 <listitem>
864 <para>The <firstterm>drag</firstterm> stage, which lasts
865 from the final use until the last reference to the object
866 is dropped.</para>
867 </listitem>
868 <listitem>
869 <para>An object which is never used is said to be in the
870 <firstterm>void</firstterm> state for its whole
871 lifetime.</para>
872 </listitem>
873 </itemizedlist>
874
875 <para>A biographical heap profile displays the portion of the
876 live heap in each of the four states listed above. Usually the
877 most interesting states are the void and drag states: live heap
878 in these states is more likely to be wasted space than heap in
879 the lag or use states.</para>
880
881 <para>It is also possible to break down the heap in one or more
882 of these states by a different criteria, by restricting a
883 profile by biography. For example, to show the portion of the
884 heap in the drag or void state by producer: </para>
885
886 <screen>
887 <replaceable>prog</replaceable> +RTS -hc -hbdrag,void
888 </screen>
889
890 <para>Once you know the producer or the type of the heap in the
891 drag or void states, the next step is usually to find the
892 retainer(s):</para>
893
894 <screen>
895 <replaceable>prog</replaceable> +RTS -hr -hc<replaceable>cc</replaceable>...
896 </screen>
897
898 <para>NOTE: this two stage process is required because GHC
899 cannot currently profile using both biographical and retainer
900 information simultaneously.</para>
901 </sect2>
902
903 <sect2 id="mem-residency">
904 <title>Actual memory residency</title>
905
906 <para>How does the heap residency reported by the heap profiler relate to
907 the actual memory residency of your program when you run it? You might
908 see a large discrepancy between the residency reported by the heap
909 profiler, and the residency reported by tools on your system
910 (eg. <literal>ps</literal> or <literal>top</literal> on Unix, or the
911 Task Manager on Windows). There are several reasons for this:</para>
912
913 <itemizedlist>
914 <listitem>
915 <para>There is an overhead of profiling itself, which is subtracted
916 from the residency figures by the profiler. This overhead goes
917 away when compiling without profiling support, of course. The
918 space overhead is currently 2 extra
919 words per heap object, which probably results in
920 about a 30% overhead.</para>
921 </listitem>
922
923 <listitem>
924 <para>Garbage collection requires more memory than the actual
925 residency. The factor depends on the kind of garbage collection
926 algorithm in use: a major GC in the standard
927 generation copying collector will usually require 3L bytes of
928 memory, where L is the amount of live data. This is because by
929 default (see the <option>+RTS -F</option> option) we allow the old
930 generation to grow to twice its size (2L) before collecting it, and
931 we require additionally L bytes to copy the live data into. When
932 using compacting collection (see the <option>+RTS -c</option>
933 option), this is reduced to 2L, and can further be reduced by
934 tweaking the <option>-F</option> option. Also add the size of the
935 allocation area (currently a fixed 512Kb).</para>
936 </listitem>
937
938 <listitem>
939 <para>The stack isn't counted in the heap profile by default. See the
940 <option>+RTS -xt</option> option.</para>
941 </listitem>
942
943 <listitem>
944 <para>The program text itself, the C stack, any non-heap data (eg. data
945 allocated by foreign libraries, and data allocated by the RTS), and
946 <literal>mmap()</literal>'d memory are not counted in the heap profile.</para>
947 </listitem>
948 </itemizedlist>
949 </sect2>
950
951 </sect1>
952
953 <sect1 id="hp2ps">
954 <title><command>hp2ps</command>&ndash;&ndash;heap profile to PostScript</title>
955
956 <indexterm><primary><command>hp2ps</command></primary></indexterm>
957 <indexterm><primary>heap profiles</primary></indexterm>
958 <indexterm><primary>postscript, from heap profiles</primary></indexterm>
959 <indexterm><primary><option>-h&lt;break-down&gt;</option></primary></indexterm>
960
961 <para>Usage:</para>
962
963 <screen>
964 hp2ps [flags] [&lt;file&gt;[.hp]]
965 </screen>
966
967 <para>The program
968 <command>hp2ps</command><indexterm><primary>hp2ps
969 program</primary></indexterm> converts a heap profile as produced
970 by the <option>-h&lt;break-down&gt;</option> runtime option into a
971 PostScript graph of the heap profile. By convention, the file to
972 be processed by <command>hp2ps</command> has a
973 <filename>.hp</filename> extension. The PostScript output is
974 written to <filename>&lt;file&gt;@.ps</filename>. If
975 <filename>&lt;file&gt;</filename> is omitted entirely, then the
976 program behaves as a filter.</para>
977
978 <para><command>hp2ps</command> is distributed in
979 <filename>ghc/utils/hp2ps</filename> in a GHC source
980 distribution. It was originally developed by Dave Wakeling as part
981 of the HBC/LML heap profiler.</para>
982
983 <para>The flags are:</para>
984
985 <variablelist>
986
987 <varlistentry>
988 <term><option>-d</option></term>
989 <listitem>
990 <para>In order to make graphs more readable,
991 <command>hp2ps</command> sorts the shaded bands for each
992 identifier. The default sort ordering is for the bands with
993 the largest area to be stacked on top of the smaller ones.
994 The <option>-d</option> option causes rougher bands (those
995 representing series of values with the largest standard
996 deviations) to be stacked on top of smoother ones.</para>
997 </listitem>
998 </varlistentry>
999
1000 <varlistentry>
1001 <term><option>-b</option></term>
1002 <listitem>
1003 <para>Normally, <command>hp2ps</command> puts the title of
1004 the graph in a small box at the top of the page. However, if
1005 the JOB string is too long to fit in a small box (more than
1006 35 characters), then <command>hp2ps</command> will choose to
1007 use a big box instead. The <option>-b</option> option
1008 forces <command>hp2ps</command> to use a big box.</para>
1009 </listitem>
1010 </varlistentry>
1011
1012 <varlistentry>
1013 <term><option>-e&lt;float&gt;[in&verbar;mm&verbar;pt]</option></term>
1014 <listitem>
1015 <para>Generate encapsulated PostScript suitable for
1016 inclusion in LaTeX documents. Usually, the PostScript graph
1017 is drawn in landscape mode in an area 9 inches wide by 6
1018 inches high, and <command>hp2ps</command> arranges for this
1019 area to be approximately centred on a sheet of a4 paper.
1020 This format is convenient of studying the graph in detail,
1021 but it is unsuitable for inclusion in LaTeX documents. The
1022 <option>-e</option> option causes the graph to be drawn in
1023 portrait mode, with float specifying the width in inches,
1024 millimetres or points (the default). The resulting
1025 PostScript file conforms to the Encapsulated PostScript
1026 (EPS) convention, and it can be included in a LaTeX document
1027 using Rokicki's dvi-to-PostScript converter
1028 <command>dvips</command>.</para>
1029 </listitem>
1030 </varlistentry>
1031
1032 <varlistentry>
1033 <term><option>-g</option></term>
1034 <listitem>
1035 <para>Create output suitable for the <command>gs</command>
1036 PostScript previewer (or similar). In this case the graph is
1037 printed in portrait mode without scaling. The output is
1038 unsuitable for a laser printer.</para>
1039 </listitem>
1040 </varlistentry>
1041
1042 <varlistentry>
1043 <term><option>-l</option></term>
1044 <listitem>
1045 <para>Normally a profile is limited to 20 bands with
1046 additional identifiers being grouped into an
1047 <literal>OTHER</literal> band. The <option>-l</option> flag
1048 removes this 20 band and limit, producing as many bands as
1049 necessary. No key is produced as it won't fit!. It is useful
1050 for creation time profiles with many bands.</para>
1051 </listitem>
1052 </varlistentry>
1053
1054 <varlistentry>
1055 <term><option>-m&lt;int&gt;</option></term>
1056 <listitem>
1057 <para>Normally a profile is limited to 20 bands with
1058 additional identifiers being grouped into an
1059 <literal>OTHER</literal> band. The <option>-m</option> flag
1060 specifies an alternative band limit (the maximum is
1061 20).</para>
1062
1063 <para><option>-m0</option> requests the band limit to be
1064 removed. As many bands as necessary are produced. However no
1065 key is produced as it won't fit! It is useful for displaying
1066 creation time profiles with many bands.</para>
1067 </listitem>
1068 </varlistentry>
1069
1070 <varlistentry>
1071 <term><option>-p</option></term>
1072 <listitem>
1073 <para>Use previous parameters. By default, the PostScript
1074 graph is automatically scaled both horizontally and
1075 vertically so that it fills the page. However, when
1076 preparing a series of graphs for use in a presentation, it
1077 is often useful to draw a new graph using the same scale,
1078 shading and ordering as a previous one. The
1079 <option>-p</option> flag causes the graph to be drawn using
1080 the parameters determined by a previous run of
1081 <command>hp2ps</command> on <filename>file</filename>. These
1082 are extracted from <filename>file@.aux</filename>.</para>
1083 </listitem>
1084 </varlistentry>
1085
1086 <varlistentry>
1087 <term><option>-s</option></term>
1088 <listitem>
1089 <para>Use a small box for the title.</para>
1090 </listitem>
1091 </varlistentry>
1092
1093 <varlistentry>
1094 <term><option>-t&lt;float&gt;</option></term>
1095 <listitem>
1096 <para>Normally trace elements which sum to a total of less
1097 than 1&percnt; of the profile are removed from the
1098 profile. The <option>-t</option> option allows this
1099 percentage to be modified (maximum 5&percnt;).</para>
1100
1101 <para><option>-t0</option> requests no trace elements to be
1102 removed from the profile, ensuring that all the data will be
1103 displayed.</para>
1104 </listitem>
1105 </varlistentry>
1106
1107 <varlistentry>
1108 <term><option>-c</option></term>
1109 <listitem>
1110 <para>Generate colour output.</para>
1111 </listitem>
1112 </varlistentry>
1113
1114 <varlistentry>
1115 <term><option>-y</option></term>
1116 <listitem>
1117 <para>Ignore marks.</para>
1118 </listitem>
1119 </varlistentry>
1120
1121 <varlistentry>
1122 <term><option>-?</option></term>
1123 <listitem>
1124 <para>Print out usage information.</para>
1125 </listitem>
1126 </varlistentry>
1127 </variablelist>
1128
1129
1130 <sect2 id="manipulating-hp">
1131 <title>Manipulating the hp file</title>
1132
1133 <para>(Notes kindly offered by Jan-Willhem Maessen.)</para>
1134
1135 <para>
1136 The <filename>FOO.hp</filename> file produced when you ask for the
1137 heap profile of a program <filename>FOO</filename> is a text file with a particularly
1138 simple structure. Here's a representative example, with much of the
1139 actual data omitted:
1140 <screen>
1141 JOB "FOO -hC"
1142 DATE "Thu Dec 26 18:17 2002"
1143 SAMPLE_UNIT "seconds"
1144 VALUE_UNIT "bytes"
1145 BEGIN_SAMPLE 0.00
1146 END_SAMPLE 0.00
1147 BEGIN_SAMPLE 15.07
1148 ... sample data ...
1149 END_SAMPLE 15.07
1150 BEGIN_SAMPLE 30.23
1151 ... sample data ...
1152 END_SAMPLE 30.23
1153 ... etc.
1154 BEGIN_SAMPLE 11695.47
1155 END_SAMPLE 11695.47
1156 </screen>
1157 The first four lines (<literal>JOB</literal>, <literal>DATE</literal>, <literal>SAMPLE_UNIT</literal>, <literal>VALUE_UNIT</literal>) form a
1158 header. Each block of lines starting with <literal>BEGIN_SAMPLE</literal> and ending
1159 with <literal>END_SAMPLE</literal> forms a single sample (you can think of this as a
1160 vertical slice of your heap profile). The hp2ps utility should accept
1161 any input with a properly-formatted header followed by a series of
1162 *complete* samples.
1163 </para>
1164 </sect2>
1165
1166 <sect2>
1167 <title>Zooming in on regions of your profile</title>
1168
1169 <para>
1170 You can look at particular regions of your profile simply by loading a
1171 copy of the <filename>.hp</filename> file into a text editor and deleting the unwanted
1172 samples. The resulting <filename>.hp</filename> file can be run through <command>hp2ps</command> and viewed
1173 or printed.
1174 </para>
1175 </sect2>
1176
1177 <sect2>
1178 <title>Viewing the heap profile of a running program</title>
1179
1180 <para>
1181 The <filename>.hp</filename> file is generated incrementally as your
1182 program runs. In principle, running <command>hp2ps</command> on the incomplete file
1183 should produce a snapshot of your program's heap usage. However, the
1184 last sample in the file may be incomplete, causing <command>hp2ps</command> to fail. If
1185 you are using a machine with UNIX utilities installed, it's not too
1186 hard to work around this problem (though the resulting command line
1187 looks rather Byzantine):
1188 <screen>
1189 head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
1190 | hp2ps > FOO.ps
1191 </screen>
1192
1193 The command <command>fgrep -n END_SAMPLE FOO.hp</command> finds the
1194 end of every complete sample in <filename>FOO.hp</filename>, and labels each sample with
1195 its ending line number. We then select the line number of the last
1196 complete sample using <command>tail</command> and <command>cut</command>. This is used as a
1197 parameter to <command>head</command>; the result is as if we deleted the final
1198 incomplete sample from <filename>FOO.hp</filename>. This results in a properly-formatted
1199 .hp file which we feed directly to <command>hp2ps</command>.
1200 </para>
1201 </sect2>
1202 <sect2>
1203 <title>Viewing a heap profile in real time</title>
1204
1205 <para>
1206 The <command>gv</command> and <command>ghostview</command> programs
1207 have a "watch file" option can be used to view an up-to-date heap
1208 profile of your program as it runs. Simply generate an incremental
1209 heap profile as described in the previous section. Run <command>gv</command> on your
1210 profile:
1211 <screen>
1212 gv -watch -seascape FOO.ps
1213 </screen>
1214 If you forget the <literal>-watch</literal> flag you can still select
1215 "Watch file" from the "State" menu. Now each time you generate a new
1216 profile <filename>FOO.ps</filename> the view will update automatically.
1217 </para>
1218
1219 <para>
1220 This can all be encapsulated in a little script:
1221 <screen>
1222 #!/bin/sh
1223 head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
1224 | hp2ps > FOO.ps
1225 gv -watch -seascape FOO.ps &amp;
1226 while [ 1 ] ; do
1227 sleep 10 # We generate a new profile every 10 seconds.
1228 head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
1229 | hp2ps > FOO.ps
1230 done
1231 </screen>
1232 Occasionally <command>gv</command> will choke as it tries to read an incomplete copy of
1233 <filename>FOO.ps</filename> (because <command>hp2ps</command> is still running as an update
1234 occurs). A slightly more complicated script works around this
1235 problem, by using the fact that sending a SIGHUP to gv will cause it
1236 to re-read its input file:
1237 <screen>
1238 #!/bin/sh
1239 head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
1240 | hp2ps > FOO.ps
1241 gv FOO.ps &amp;
1242 gvpsnum=$!
1243 while [ 1 ] ; do
1244 sleep 10
1245 head -`fgrep -n END_SAMPLE FOO.hp | tail -1 | cut -d : -f 1` FOO.hp \
1246 | hp2ps > FOO.ps
1247 kill -HUP $gvpsnum
1248 done
1249 </screen>
1250 </para>
1251 </sect2>
1252 </sect1>
1253
1254 <sect1 id="hpc">
1255 <title>Observing Code Coverage</title>
1256 <indexterm><primary>code coverage</primary></indexterm>
1257 <indexterm><primary>Haskell Program Coverage</primary></indexterm>
1258 <indexterm><primary>hpc</primary></indexterm>
1259
1260 <para>
1261 Code coverage tools allow a programmer to determine what parts of
1262 their code have been actually executed, and which parts have
1263 never actually been invoked. GHC has an option for generating
1264 instrumented code that records code coverage as part of the
1265 <ulink url="http://www.haskell.org/hpc">Haskell Program Coverage
1266 </ulink>(HPC) toolkit, which is included with GHC. HPC tools can
1267 be used to render the generated code coverage information into
1268 human understandable format. </para>
1269
1270 <para>
1271 Correctly instrumented code provides coverage information of two
1272 kinds: source coverage and boolean-control coverage. Source
1273 coverage is the extent to which every part of the program was
1274 used, measured at three different levels: declarations (both
1275 top-level and local), alternatives (among several equations or
1276 case branches) and expressions (at every level). Boolean
1277 coverage is the extent to which each of the values True and
1278 False is obtained in every syntactic boolean context (ie. guard,
1279 condition, qualifier). </para>
1280
1281 <para>
1282 HPC displays both kinds of information in two primary ways:
1283 textual reports with summary statistics (hpc report) and sources
1284 with color mark-up (hpc markup). For boolean coverage, there
1285 are four possible outcomes for each guard, condition or
1286 qualifier: both True and False values occur; only True; only
1287 False; never evaluated. In hpc-markup output, highlighting with
1288 a yellow background indicates a part of the program that was
1289 never evaluated; a green background indicates an always-True
1290 expression and a red background indicates an always-False one.
1291 </para>
1292
1293 <sect2><title>A small example: Reciprocation</title>
1294
1295 <para>
1296 For an example we have a program, called Recip.hs, which computes exact decimal
1297 representations of reciprocals, with recurring parts indicated in
1298 brackets.
1299 </para>
1300 <programlisting>
1301 reciprocal :: Int -> (String, Int)
1302 reciprocal n | n > 1 = ('0' : '.' : digits, recur)
1303 | otherwise = error
1304 "attempting to compute reciprocal of number &lt;= 1"
1305 where
1306 (digits, recur) = divide n 1 []
1307 divide :: Int -> Int -> [Int] -> (String, Int)
1308 divide n c cs | c `elem` cs = ([], position c cs)
1309 | r == 0 = (show q, 0)
1310 | r /= 0 = (show q ++ digits, recur)
1311 where
1312 (q, r) = (c*10) `quotRem` n
1313 (digits, recur) = divide n r (c:cs)
1314
1315 position :: Int -> [Int] -> Int
1316 position n (x:xs) | n==x = 1
1317 | otherwise = 1 + position n xs
1318
1319 showRecip :: Int -> String
1320 showRecip n =
1321 "1/" ++ show n ++ " = " ++
1322 if r==0 then d else take p d ++ "(" ++ drop p d ++ ")"
1323 where
1324 p = length d - r
1325 (d, r) = reciprocal n
1326
1327 main = do
1328 number &lt;- readLn
1329 putStrLn (showRecip number)
1330 main
1331 </programlisting>
1332
1333 <para>The HPC instrumentation is enabled using the -fhpc flag.
1334 </para>
1335
1336 <screen>
1337 $ ghc -fhpc Recip.hs --make
1338 </screen>
1339 <para>HPC index (.mix) files are placed placed in .hpc subdirectory. These can be considered like
1340 the .hi files for HPC.
1341 </para>
1342 <screen>
1343 $ ./Recip
1344 1/3
1345 = 0.(3)
1346 </screen>
1347 <para>We can generate a textual summary of coverage:</para>
1348 <screen>
1349 $ hpc report Recip
1350 80% expressions used (81/101)
1351 12% boolean coverage (1/8)
1352 14% guards (1/7), 3 always True,
1353 1 always False,
1354 2 unevaluated
1355 0% 'if' conditions (0/1), 1 always False
1356 100% qualifiers (0/0)
1357 55% alternatives used (5/9)
1358 100% local declarations used (9/9)
1359 100% top-level declarations used (5/5)
1360 </screen>
1361 <para>We can also generate a marked-up version of the source.</para>
1362 <screen>
1363 $ hpc markup Recip
1364 writing Recip.hs.html
1365 </screen>
1366 <para>
1367 This generates one file per Haskell module, and 4 index files,
1368 hpc_index.html, hpc_index_alt.html, hpc_index_exp.html,
1369 hpc_index_fun.html.
1370 </para>
1371 </sect2>
1372
1373 <sect2><title>Options for instrumenting code for coverage</title>
1374 <para>
1375 Turning on code coverage is easy, use the -fhpc flag.
1376 Instrumented and non-instrumented can be freely mixed.
1377 When compiling the Main module GHC automatically detects when there
1378 is an hpc compiled file, and adds the correct initialization code.
1379 </para>
1380
1381 </sect2>
1382
1383 <sect2><title>The hpc toolkit</title>
1384
1385 <para>
1386 The hpc toolkit uses a cvs/svn/darcs-like interface, where a
1387 single binary contains many function units.</para>
1388 <screen>
1389 $ hpc
1390 Usage: hpc COMMAND ...
1391
1392 Commands:
1393 help Display help for hpc or a single command
1394 Reporting Coverage:
1395 report Output textual report about program coverage
1396 markup Markup Haskell source with program coverage
1397 Processing Coverage files:
1398 sum Sum multiple .tix files in a single .tix file
1399 combine Combine two .tix files in a single .tix file
1400 map Map a function over a single .tix file
1401 Coverage Overlays:
1402 overlay Generate a .tix file from an overlay file
1403 draft Generate draft overlay that provides 100% coverage
1404 Others:
1405 show Show .tix file in readable, verbose format
1406 version Display version for hpc
1407 </screen>
1408
1409 <para>In general, these options act on .tix file after an
1410 instrumented binary has generated it, which hpc acting as a
1411 conduit between the raw .tix file, and the more detailed reports
1412 produced.
1413 </para>
1414
1415 <para>
1416 The hpc tool assumes you are in the top-level directory of
1417 the location where you built your application, and the .tix
1418 file is in the same top-level directory. You can use the
1419 flag --srcdir to use hpc for any other directory, and use
1420 --srcdir multiple times to analyse programs compiled from
1421 difference locations, as is typical for packages.
1422 </para>
1423
1424 <para>
1425 We now explain in more details the major modes of hpc.
1426 </para>
1427
1428 <sect3><title>hpc report</title>
1429 <para>hpc report gives a textual report of coverage. By default,
1430 all modules and packages are considered in generating report,
1431 unless include or exclude are used. The report is a summary
1432 unless the --per-module flag is used. The --xml-output option
1433 allows for tools to use hpc to glean coverage.
1434 </para>
1435 <screen>
1436 $ hpc help report
1437 Usage: hpc report [OPTION] .. &lt;TIX_FILE&gt; [&lt;MODULE&gt; [&lt;MODULE&gt; ..]]
1438
1439 Options:
1440
1441 --per-module show module level detail
1442 --decl-list show unused decls
1443 --exclude=[PACKAGE:][MODULE] exclude MODULE and/or PACKAGE
1444 --include=[PACKAGE:][MODULE] include MODULE and/or PACKAGE
1445 --srcdir=DIR path to source directory of .hs files
1446 multi-use of srcdir possible
1447 --hpcdir=DIR sub-directory that contains .mix files
1448 default .hpc [rarely used]
1449 --xml-output show output in XML
1450 </screen>
1451 </sect3>
1452 <sect3><title>hpc markup</title>
1453 <para>hpc markup marks up source files into colored html.
1454 </para>
1455 <screen>
1456 $ hpc help markup
1457 Usage: hpc markup [OPTION] .. &lt;TIX_FILE&gt; [&lt;MODULE&gt; [&lt;MODULE&gt; ..]]
1458
1459 Options:
1460
1461 --exclude=[PACKAGE:][MODULE] exclude MODULE and/or PACKAGE
1462 --include=[PACKAGE:][MODULE] include MODULE and/or PACKAGE
1463 --srcdir=DIR path to source directory of .hs files
1464 multi-use of srcdir possible
1465 --hpcdir=DIR sub-directory that contains .mix files
1466 default .hpc [rarely used]
1467 --fun-entry-count show top-level function entry counts
1468 --highlight-covered highlight covered code, rather that code gaps
1469 --destdir=DIR path to write output to
1470 </screen>
1471
1472 </sect3>
1473 <sect3><title>hpc sum</title>
1474 <para>hpc sum adds together any number of .tix files into a single
1475 .tix file. hpc sum does not change the original .tix file; it generates a new .tix file.
1476 </para>
1477 <screen>
1478 $ hpc help sum
1479 Usage: hpc sum [OPTION] .. &lt;TIX_FILE&gt; [&lt;TIX_FILE&gt; [&lt;TIX_FILE&gt; ..]]
1480 Sum multiple .tix files in a single .tix file
1481
1482 Options:
1483
1484 --exclude=[PACKAGE:][MODULE] exclude MODULE and/or PACKAGE
1485 --include=[PACKAGE:][MODULE] include MODULE and/or PACKAGE
1486 --output=FILE output FILE
1487 --union use the union of the module namespace (default is intersection)
1488 </screen>
1489 </sect3>
1490 <sect3><title>hpc combine</title>
1491 <para>hpc combine is the swiss army knife of hpc. It can be
1492 used to take the difference between .tix files, to subtract one
1493 .tix file from another, or to add two .tix files. hpc combine does not
1494 change the original .tix file; it generates a new .tix file.
1495 </para>
1496 <screen>
1497 $ hpc help combine
1498 Usage: hpc combine [OPTION] .. &lt;TIX_FILE&gt; &lt;TIX_FILE&gt;
1499 Combine two .tix files in a single .tix file
1500
1501 Options:
1502
1503 --exclude=[PACKAGE:][MODULE] exclude MODULE and/or PACKAGE
1504 --include=[PACKAGE:][MODULE] include MODULE and/or PACKAGE
1505 --output=FILE output FILE
1506 --function=FUNCTION combine .tix files with join function, default = ADD
1507 FUNCTION = ADD | DIFF | SUB
1508 --union use the union of the module namespace (default is intersection)
1509 </screen>
1510 </sect3>
1511 <sect3><title>hpc map</title>
1512 <para>hpc map inverts or zeros a .tix file. hpc map does not
1513 change the original .tix file; it generates a new .tix file.
1514 </para>
1515 <screen>
1516 $ hpc help map
1517 Usage: hpc map [OPTION] .. &lt;TIX_FILE&gt;
1518 Map a function over a single .tix file
1519
1520 Options:
1521
1522 --exclude=[PACKAGE:][MODULE] exclude MODULE and/or PACKAGE
1523 --include=[PACKAGE:][MODULE] include MODULE and/or PACKAGE
1524 --output=FILE output FILE
1525 --function=FUNCTION apply function to .tix files, default = ID
1526 FUNCTION = ID | INV | ZERO
1527 --union use the union of the module namespace (default is intersection)
1528 </screen>
1529 </sect3>
1530 <sect3><title>hpc overlay and hpc draft</title>
1531 <para>
1532 Overlays are an experimental feature of HPC, a textual description
1533 of coverage. hpc draft is used to generate a draft overlay from a .tix file,
1534 and hpc overlay generates a .tix files from an overlay.
1535 </para>
1536 <screen>
1537 % hpc help overlay
1538 Usage: hpc overlay [OPTION] .. &lt;OVERLAY_FILE&gt; [&lt;OVERLAY_FILE&gt; [...]]
1539
1540 Options:
1541
1542 --srcdir=DIR path to source directory of .hs files
1543 multi-use of srcdir possible
1544 --hpcdir=DIR sub-directory that contains .mix files
1545 default .hpc [rarely used]
1546 --output=FILE output FILE
1547 % hpc help draft
1548 Usage: hpc draft [OPTION] .. &lt;TIX_FILE&gt;
1549
1550 Options:
1551
1552 --exclude=[PACKAGE:][MODULE] exclude MODULE and/or PACKAGE
1553 --include=[PACKAGE:][MODULE] include MODULE and/or PACKAGE
1554 --srcdir=DIR path to source directory of .hs files
1555 multi-use of srcdir possible
1556 --hpcdir=DIR sub-directory that contains .mix files
1557 default .hpc [rarely used]
1558 --output=FILE output FILE
1559 </screen>
1560 </sect3>
1561 </sect2>
1562 <sect2><title>Caveats and Shortcomings of Haskell Program Coverage</title>
1563 <para>
1564 HPC does not attempt to lock the .tix file, so multiple concurrently running
1565 binaries in the same directory will exhibit a race condition. There is no way
1566 to change the name of the .tix file generated, apart from renaming the binary.
1567 HPC does not work with GHCi.
1568 </para>
1569 </sect2>
1570 </sect1>
1571
1572 <sect1 id="ticky-ticky">
1573 <title>Using &ldquo;ticky-ticky&rdquo; profiling (for implementors)</title>
1574 <indexterm><primary>ticky-ticky profiling</primary></indexterm>
1575
1576 <para>(ToDo: document properly.)</para>
1577
1578 <para>It is possible to compile Glasgow Haskell programs so that
1579 they will count lots and lots of interesting things, e.g., number
1580 of updates, number of data constructors entered, etc., etc. We
1581 call this &ldquo;ticky-ticky&rdquo;
1582 profiling,<indexterm><primary>ticky-ticky
1583 profiling</primary></indexterm> <indexterm><primary>profiling,
1584 ticky-ticky</primary></indexterm> because that's the sound a Sun4
1585 makes when it is running up all those counters
1586 (<emphasis>slowly</emphasis>).</para>
1587
1588 <para>Ticky-ticky profiling is mainly intended for implementors;
1589 it is quite separate from the main &ldquo;cost-centre&rdquo;
1590 profiling system, intended for all users everywhere.</para>
1591
1592 <para>To be able to use ticky-ticky profiling, you will need to
1593 have built the ticky RTS. (This should be described in
1594 the building guide, but amounts to building the RTS with way
1595 "t" enabled.)</para>
1596
1597 <para>To get your compiled program to spit out the ticky-ticky
1598 numbers, use a <option>-r</option> RTS
1599 option<indexterm><primary>-r RTS option</primary></indexterm>.
1600 See <xref linkend="runtime-control"/>.</para>
1601
1602 <para>Compiling your program with the <option>-ticky</option>
1603 switch yields an executable that performs these counts. Here is a
1604 sample ticky-ticky statistics file, generated by the invocation
1605 <command>foo +RTS -rfoo.ticky</command>.</para>
1606
1607 <screen>
1608 foo +RTS -rfoo.ticky
1609
1610
1611 ALLOCATIONS: 3964631 (11330900 words total: 3999476 admin, 6098829 goods, 1232595 slop)
1612 total words: 2 3 4 5 6+
1613 69647 ( 1.8%) function values 50.0 50.0 0.0 0.0 0.0
1614 2382937 ( 60.1%) thunks 0.0 83.9 16.1 0.0 0.0
1615 1477218 ( 37.3%) data values 66.8 33.2 0.0 0.0 0.0
1616 0 ( 0.0%) big tuples
1617 2 ( 0.0%) black holes 0.0 100.0 0.0 0.0 0.0
1618 0 ( 0.0%) prim things
1619 34825 ( 0.9%) partial applications 0.0 0.0 0.0 100.0 0.0
1620 2 ( 0.0%) thread state objects 0.0 0.0 0.0 0.0 100.0
1621
1622 Total storage-manager allocations: 3647137 (11882004 words)
1623 [551104 words lost to speculative heap-checks]
1624
1625 STACK USAGE:
1626
1627 ENTERS: 9400092 of which 2005772 (21.3%) direct to the entry code
1628 [the rest indirected via Node's info ptr]
1629 1860318 ( 19.8%) thunks
1630 3733184 ( 39.7%) data values
1631 3149544 ( 33.5%) function values
1632 [of which 1999880 (63.5%) bypassed arg-satisfaction chk]
1633 348140 ( 3.7%) partial applications
1634 308906 ( 3.3%) normal indirections
1635 0 ( 0.0%) permanent indirections
1636
1637 RETURNS: 5870443
1638 2137257 ( 36.4%) from entering a new constructor
1639 [the rest from entering an existing constructor]
1640 2349219 ( 40.0%) vectored [the rest unvectored]
1641
1642 RET_NEW: 2137257: 32.5% 46.2% 21.3% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
1643 RET_OLD: 3733184: 2.8% 67.9% 29.3% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
1644 RET_UNBOXED_TUP: 2: 0.0% 0.0%100.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
1645
1646 RET_VEC_RETURN : 2349219: 0.0% 0.0%100.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
1647
1648 UPDATE FRAMES: 2241725 (0 omitted from thunks)
1649 SEQ FRAMES: 1
1650 CATCH FRAMES: 1
1651 UPDATES: 2241725
1652 0 ( 0.0%) data values
1653 34827 ( 1.6%) partial applications
1654 [2 in place, 34825 allocated new space]
1655 2206898 ( 98.4%) updates to existing heap objects (46 by squeezing)
1656 UPD_CON_IN_NEW: 0: 0 0 0 0 0 0 0 0 0
1657 UPD_PAP_IN_NEW: 34825: 0 0 0 34825 0 0 0 0 0
1658
1659 NEW GEN UPDATES: 2274700 ( 99.9%)
1660
1661 OLD GEN UPDATES: 1852 ( 0.1%)
1662
1663 Total bytes copied during GC: 190096
1664
1665 **************************************************
1666 3647137 ALLOC_HEAP_ctr
1667 11882004 ALLOC_HEAP_tot
1668 69647 ALLOC_FUN_ctr
1669 69647 ALLOC_FUN_adm
1670 69644 ALLOC_FUN_gds
1671 34819 ALLOC_FUN_slp
1672 34831 ALLOC_FUN_hst_0
1673 34816 ALLOC_FUN_hst_1
1674 0 ALLOC_FUN_hst_2
1675 0 ALLOC_FUN_hst_3
1676 0 ALLOC_FUN_hst_4
1677 2382937 ALLOC_UP_THK_ctr
1678 0 ALLOC_SE_THK_ctr
1679 308906 ENT_IND_ctr
1680 0 E!NT_PERM_IND_ctr requires +RTS -Z
1681 [... lots more info omitted ...]
1682 0 GC_SEL_ABANDONED_ctr
1683 0 GC_SEL_MINOR_ctr
1684 0 GC_SEL_MAJOR_ctr
1685 0 GC_FAILED_PROMOTION_ctr
1686 47524 GC_WORDS_COPIED_ctr
1687 </screen>
1688
1689 <para>The formatting of the information above the row of asterisks
1690 is subject to change, but hopefully provides a useful
1691 human-readable summary. Below the asterisks <emphasis>all
1692 counters</emphasis> maintained by the ticky-ticky system are
1693 dumped, in a format intended to be machine-readable: zero or more
1694 spaces, an integer, a space, the counter name, and a newline.</para>
1695
1696 <para>In fact, not <emphasis>all</emphasis> counters are
1697 necessarily dumped; compile- or run-time flags can render certain
1698 counters invalid. In this case, either the counter will simply
1699 not appear, or it will appear with a modified counter name,
1700 possibly along with an explanation for the omission (notice
1701 <literal>ENT&lowbar;PERM&lowbar;IND&lowbar;ctr</literal> appears
1702 with an inserted <literal>!</literal> above). Software analysing
1703 this output should always check that it has the counters it
1704 expects. Also, beware: some of the counters can have
1705 <emphasis>large</emphasis> values!</para>
1706
1707 </sect1>
1708
1709 </chapter>
1710
1711 <!-- Emacs stuff:
1712 ;;; Local Variables: ***
1713 ;;; mode: xml ***
1714 ;;; sgml-parent-document: ("users_guide.xml" "book" "chapter") ***
1715 ;;; End: ***
1716 -->