You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-13 - 13:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot8s.osadl.org (updated Thu Nov 13, 2025 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25947439912380,1235cyclictest27243872sleep320:49:563
25947439911720,1169cyclictest28722762sleep322:25:003
25947439910850,1081cyclictest29863282sleep323:35:023
2594743992420,2cyclictest2737075-21turbostat21:00:003
2594743992218,1cyclictest0-21swapper/321:35:013
26477972192,5sleep12647819-21apt-get19:50:141
2594743991815,1cyclictest0-21swapper/321:10:013
2594743991815,1cyclictest0-21swapper/300:00:003
259474399176,8cyclictest2757892-21taskset21:15:013
259474399167,6cyclictest2766016-21turbostat21:20:013
2594743991614,1cyclictest0-21swapper/320:15:193
2594743991613,1cyclictest0-21swapper/323:00:173
2594743991613,1cyclictest0-21swapper/322:45:203
2594743991613,1cyclictest0-21swapper/321:54:483
2594743991613,1cyclictest0-21swapper/320:30:143
259474399161,11cyclictest2837103-21taskset22:00:253
132298160,4rtkit-daemon2870284-21ssh22:20:521
259474399156,7cyclictest26862732sleep320:20:133
259474399155,7cyclictest29449852sleep323:09:343
259474399154,9cyclictest2945422-21taskset23:10:013
2594743991513,0cyclictest0-21swapper/319:40:473
2594743991513,0cyclictest0-21swapper/319:35:383
2594743991513,0cyclictest0-21swapper/319:13:383
2594743991512,1cyclictest0-21swapper/322:24:123
2594743991512,1cyclictest0-21swapper/322:10:003
2594743991512,1cyclictest0-21swapper/320:27:503
2594743991512,1cyclictest0-21swapper/319:56:393
2594743991512,1cyclictest0-21swapper/319:45:443
259474399151,11cyclictest3084007-21taskset00:35:003
259474399151,11cyclictest3050274-21chrt00:12:443
259474399151,11cyclictest2829110-21taskset21:55:243
259474399151,11cyclictest2774341-21taskset21:25:023
259474399151,10cyclictest2893933-21taskset22:35:233
259474399151,10cyclictest2856445-21chrt22:15:153
259474399151,10cyclictest2667220-21taskset20:05:133
30857112141,11sleep03087417-21cut00:35:180
259474399145,7cyclictest2737075-21turbostat21:04:553
259474399144,7cyclictest3066795-21taskset00:23:103
2594743991412,1cyclictest0-21swapper/322:41:063
2594743991412,0cyclictest0-21swapper/319:50:423
2594743991411,1cyclictest0-21swapper/320:10:393
259474399141,10cyclictest3073949-21taskset00:25:513
259474399141,10cyclictest3032310-21taskset00:00:183
259474399141,10cyclictest2994728-21taskset23:40:093
259474399141,10cyclictest2960910-21chrt23:18:513
259474399141,10cyclictest2880424-21taskset22:30:103
259474399141,10cyclictest2853882-21taskset22:10:393
259474399141,10cyclictest2812447-21chrt21:45:213
259474399141,10cyclictest2796583-21taskset21:35:243
259474399141,10cyclictest2769362-21taskset21:20:173
259474399141,10cyclictest2666296-21chrt20:00:243
259474399141,10cyclictest2628391-21taskset19:33:443
259474399141,10cyclictest2619940-21taskset19:25:233
259474399140,11cyclictest2711711-21turbostat20:40:003
259474399140,11cyclictest2615549-21taskset19:21:423
26673712130,6sleep02667486-21perl20:05:150
259474399131,9cyclictest2912945-21chrt22:50:063
259474399131,9cyclictest2912945-21chrt22:50:063
2594743991312,0cyclictest0-21swapper/320:40:493
259474399131,10cyclictest3057615-21taskset00:15:493
259474399131,10cyclictest2927596-21taskset22:56:513
259474399131,10cyclictest2805289-21chrt21:41:213
259474399131,10cyclictest2730410-21taskset20:50:533
2594743991311,0cyclictest0-21swapper/323:25:363
2594743991310,1cyclictest0-21swapper/323:20:253
259474399130,10cyclictest3090296-21chrt00:35:553
259474399130,10cyclictest3035300-21chrt00:05:033
259474399130,10cyclictest3010884-21taskset23:50:123
259474399130,10cyclictest3009735-21chrt23:48:043
259474399130,10cyclictest2983661-21chrt23:30:253
25943372130,3sleep30-21swapper/319:06:473
259474399120,9cyclictest2603782-21chrt19:15:133
132298120,8rtkit-daemon507-21gmain19:46:372
132298120,7rtkit-daemon2603791-21idleruntime19:15:132
132298120,4rtkit-daemon2601197-21munin-node20:17:322
30678942113,5sleep13067931-21apt-get00:25:131
132298110,4rtkit-daemon461-21wpa_supplicant20:22:552
132298110,4rtkit-daemon3073767-21ssh00:25:280
132298110,4rtkit-daemon2813437-21ssh21:46:340
132298110,2rtkit-daemon0-21swapper/000:20:180
29852002100,1sleep22985208-21ssh23:32:572
29687252100,1sleep20-21swapper/223:23:072
28642552100,3sleep02864415-21df_inode22:20:140
28546122101,7sleep02854613-21sh22:12:210
26671842101,5sleep22667217-21apt-get20:05:132
26541362102,4sleep12654158-21apt-get19:55:101
132298100,7rtkit-daemon2615356-21unixbench_multi19:20:232
132298100,5rtkit-daemon2737359-21apt-get21:00:122
132298100,3rtkit-daemon647-21snmpd21:59:531
132298100,3rtkit-daemon507-21gmain21:55:572
132298100,3rtkit-daemon2935735-21ssh23:01:530
3043520291,5sleep03043684-21/usr/sbin/munin00:10:140
2986406290,6sleep22986449-21ssh23:35:142
2976842290,3sleep02976846-21ssh23:28:060
2905331290,2sleep12905421-21irqcore22:45:201
2887174290,1sleep22749768-21diskmemload22:32:342
2797261291,6sleep02797269-21ssh21:36:420
2775811290,1sleep22775941-21kthreadcore21:25:182
2766341291,4sleep12766374-21apt-get21:20:131
2724044296,2sleep02641821-21kworker/0:2+kacpid20:45:340
2628747291,4sleep02628780-21apt-get19:35:140
13229890,6rtkit-daemon2601197-21munin-node21:44:020
13229890,5rtkit-daemon647-21snmpd23:15:300
13229890,5rtkit-daemon461-21wpa_supplicant21:54:202
13229890,5rtkit-daemon2660309-21sendmail-msp20:00:002
13229890,4rtkit-daemon461-21wpa_supplicant20:39:262
13229890,4rtkit-daemon461-21wpa_supplicant00:18:170
13229890,4rtkit-daemon3025917-21ssh23:57:580
13229890,4rtkit-daemon2938000-21sed23:05:160
13229890,4rtkit-daemon2880441-21ssh22:30:131
13229890,4rtkit-daemon2763506-21ntp_states21:15:192
13229890,1rtkit-daemon0-21swapper/223:28:542
3078779280,6sleep23080858-21cut00:30:192
3041210280,1sleep20-21swapper/200:05:272
3035400281,4sleep03035414-21/usr/sbin/munin00:05:130
3034660281,4sleep13034668-21ssh00:04:141
3001369280,6sleep03001377-21ssh23:42:290
2995452280,5sleep22996397-21kthreadcore23:40:142
2932911280,1sleep20-21swapper/223:00:192
2888595280,5sleep22888558-21/usr/sbin/munin22:35:152
2805315280,1sleep20-21swapper/221:41:252
2698981281,3sleep22699013-21apt-get20:30:132
2687358280,5sleep02687763-21kthreadcore20:20:180
2674108282,4sleep22675040-21grep20:10:182
2672594280,2sleep10-21swapper/120:05:241
2635140280,6sleep02635175-21grep19:40:150
2596348282,3sleep02597487-21grep19:10:180
2590809280,5sleep22591619-21kthreadcore19:05:192
13229880,5rtkit-daemon507-21gmain19:31:452
13229880,5rtkit-daemon2601197-21munin-node20:44:052
13229880,4rtkit-daemon648-21ntpd19:40:162
13229880,4rtkit-daemon461-21wpa_supplicant23:50:160
13229880,4rtkit-daemon461-21wpa_supplicant23:42:151
13229880,4rtkit-daemon461-21wpa_supplicant22:44:321
13229880,4rtkit-daemon461-21wpa_supplicant20:54:272
13229880,4rtkit-daemon2817099-21grep21:50:170
13229880,4rtkit-daemon2718556-21tune2fs20:45:142
13229880,4rtkit-daemon2601197-21munin-node23:45:571
13229880,4rtkit-daemon2601197-21munin-node22:50:271
13229880,4rtkit-daemon2601197-21munin-node22:50:261
13229880,4rtkit-daemon2601197-21munin-node22:36:451
13229880,4rtkit-daemon2601197-21munin-node21:30:111
13229880,4rtkit-daemon2601197-21munin-node20:56:232
13229880,4rtkit-daemon2601197-21munin-node19:50:382
13229880,4rtkit-daemon2601197-21munin-node19:37:532
13229880,4rtkit-daemon1415-21sendmail-mta23:23:201
13229880,4rtkit-daemon0-21swapper/220:01:132
13229880,1rtkit-daemon0-21swapper/220:28:252
13229880,1rtkit-daemon0-21swapper/122:28:181
13229880,1rtkit-daemon0-21swapper/122:16:261
13229880,1rtkit-daemon0-21swapper/122:13:221
13229880,1rtkit-daemon0-21swapper/122:00:441
3081771270,4sleep03081951-21vmstat00:30:250
3077569270,4sleep13078778-21grep00:30:181
3066266270,2sleep13066274-21ssh00:21:571
3053081271,3sleep23054633-21kthreadcore00:15:172
3033601270,1sleep20-21swapper/200:01:462
3028004271,3sleep03028940-21cut00:00:160
3021136270,4sleep10-21swapper/123:55:201
3016405271,4sleep10-21swapper/123:50:231
2983779271,3sleep00-21swapper/023:30:270
2954517270,5sleep12955401-21grep23:15:171
2945501270,4sleep22945650-21df23:10:132
2920791270,4sleep0443-21dbus-daemon22:55:000
2920791270,4sleep0443-21dbus-daemon22:55:000
2904077270,5sleep02904085-21ssh22:43:540
2846199270,5sleep12846207-21ssh22:07:131
2846053270,5sleep22846059-21ssh22:06:462
2836805270,4sleep02836866-21ntp_states22:00:230
2821551270,5sleep12821557-21ssh21:51:381
2796247271,3sleep12796373-21ssh21:35:221
2745589270,4sleep12747678-21cut21:05:181
2737788271,3sleep02738779-21irqstats21:00:160
2711968271,3sleep02712001-21apt-get20:40:130
2630201270,4sleep12631609-21kthreadcore19:35:181
2610120272,3sleep12610547-21irqcore19:20:171
2601545270,4sleep02602526-21grep19:15:030
2588552270,2sleep00-21swapper/019:05:130
13229870,4rtkit-daemon507-21gmain23:10:530
13229870,4rtkit-daemon507-21gmain22:58:410
13229870,4rtkit-daemon2601197-21munin-node22:55:011
13229870,4rtkit-daemon2556410-21munin-node19:14:182
13229870,3rtkit-daemon647-21snmpd23:49:190
13229870,3rtkit-daemon507-21gmain21:05:132
13229870,3rtkit-daemon2781824-21ssh21:29:071
13229870,3rtkit-daemon2750388-21gltestperf21:10:162
13229870,3rtkit-daemon2601197-21munin-node21:34:140
13229870,3rtkit-daemon1415-21sendmail-mta23:21:150
13229870,1rtkit-daemon0-21swapper/219:27:332
3091589260,1sleep20-21swapper/200:38:562
3084424260,3sleep13084539-21ssh00:35:151
3065026260,3sleep23065106-21ntpq00:20:222
3058366260,4sleep13058373-21sh00:17:321
3041682260,1sleep10-21swapper/100:06:341
3016896260,1sleep20-21swapper/223:50:482
2992948260,1sleep00-21swapper/023:36:560
2992561260,4sleep12992569-21ssh23:36:011
2978344260,3sleep10-21swapper/123:30:151
2951895260,1sleep10-21swapper/123:11:441
2942826261,3sleep22942951-21sed23:05:242
2939335261,2sleep10-21swapper/123:05:191
2926975260,1sleep20-21swapper/222:55:262
2918337260,3sleep22918469-21python322:50:222
2918337260,3sleep22918469-21python322:50:222
2911804260,1sleep20-21swapper/222:47:572
2905123260,3sleep02905330-21hddtemp_smartct22:45:190
2897097260,3sleep22897821-21sed22:40:152
2896106260,1sleep00-21swapper/022:39:260
2885661260,1sleep00-21swapper/022:30:230
2878319260,4sleep02878321-21sshd22:25:430
2878312260,1sleep20-21swapper/222:25:412
2861362261,2sleep00-21swapper/022:15:210
2849448261,2sleep20-21swapper/222:10:182
2847416260,3sleep02847753-21apt-config22:10:000
2838467260,1sleep20-21swapper/222:03:042
2814031260,1sleep20-21swapper/221:47:582
2764205260,1sleep10-21swapper/121:15:481
2755979260,1sleep10-21swapper/121:10:431
2724848260,3sleep12725047-21hddtemp_smartct20:50:151
2723694260,3sleep12723824-21sed20:45:231
2710795260,3sleep02710854-21ntp_states20:35:240
2653283262,2sleep02653500-21users19:50:240
2637024260,3sleep12639118-21cut19:40:201
2621567260,4sleep02621784-21users19:25:290
2616100260,4sleep12616139-21cpu19:25:171
2589550260,3sleep12590807-21kthreadcore19:05:181
13229860,3rtkit-daemon1415-21sendmail-mta23:47:502
13229860,2rtkit-daemon0-21swapper/221:20:072
3043891250,2sleep20-21swapper/200:10:162
3025573250,3sleep20-21swapper/223:57:122
2976541250,1sleep10-21swapper/123:27:191
2953564250,3sleep20-21swapper/223:15:012
2930954250,2sleep10-21swapper/123:00:181
2870373250,3sleep20-21swapper/222:21:042
2862177250,3sleep20-21swapper/222:16:032
2824815251,2sleep00-21swapper/021:55:180
2812985250,3sleep12812993-21ssh21:45:291
2805089250,3sleep10-21swapper/121:40:561
2797741250,3sleep20-21swapper/221:37:492
2782753251,2sleep20-21swapper/221:30:192
2758109250,2sleep02758117-21ssh21:15:030
2757780251,2sleep00-21swapper/021:14:500
2749736250,3sleep00-21swapper/021:09:500
2736907250,1sleep00-21swapper/020:58:120
2736846250,1sleep10-21swapper/120:57:241
2712004250,3sleep12712153-21df_inode20:40:141
2707068251,2sleep10-21swapper/120:35:211
2704876250,1sleep00-21swapper/020:32:190
2688546250,3sleep12690664-21cut20:20:191
2685709250,1sleep00-21swapper/020:16:210
2679360250,1sleep10-21swapper/120:11:331
2666537250,1sleep10-21swapper/120:03:241
2662201250,3sleep02664231-21grep20:00:180
2654519250,2sleep00-21swapper/019:55:130
2643366250,3sleep12645459-21cut19:45:211
3069385240,2sleep23070784-21kthreadcore00:25:192
3049554240,3sleep10-21swapper/100:10:581
2779128240,2sleep00-21swapper/021:25:200
2771358240,1sleep00-21swapper/021:20:200
2726641240,2sleep00-21swapper/020:50:190
2704852240,3sleep10-21swapper/120:32:001
2695171240,2sleep10-21swapper/120:25:231
2694043240,2sleep00-21swapper/020:25:220
2680085240,1sleep10-21swapper/120:15:171
2642409240,1sleep00-21swapper/019:45:200
2623892241,1sleep00-21swapper/019:30:170
2609919240,1sleep00-21swapper/019:20:160
2605692240,1sleep10-21swapper/119:15:161
2597488240,2sleep10-21swapper/119:10:191
2737362230,1sleep10-21swapper/121:00:131
2673742230,1sleep00-21swapper/020:10:160
2625879230,1sleep10-21swapper/119:30:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional