You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-18 - 10:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot8s.osadl.org (updated Thu Dec 18, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25771339912250,1221cyclictest27022302sleep320:45:023
25771339911210,1117cyclictest27343812sleep321:10:023
25771339911190,1115cyclictest27667272sleep321:35:023
2577133993580,355cyclictest2856689-21turbostat22:49:553
2577133992620,259cyclictest2824539-21turbostat22:24:543
2577133992500,246cyclictest2889018-21turbostat23:14:553
301798200,7rtkit-daemon2583607-21munin-node00:35:311
2577133992017,1cyclictest0-21swapper/321:30:013
2577133991916,1cyclictest0-21swapper/319:49:593
257713399171,11cyclictest2901843-21turbostat23:20:003
26055782160,1sleep10-21swapper/119:30:151
257713399168,5cyclictest2966463-21turbostat00:10:003
257713399167,7cyclictest26768672sleep320:25:143
257713399165,7cyclictest27412742sleep321:15:153
2577133991614,1cyclictest0-21swapper/321:50:143
2577133991614,0cyclictest0-21swapper/322:39:593
257713399161,11cyclictest2927866-21turbostat23:40:013
257713399161,11cyclictest2805286-21sleep22:05:003
2577133991513,1cyclictest0-21swapper/323:25:153
2577133991513,1cyclictest0-21swapper/322:50:173
2577133991512,1cyclictest0-21swapper/323:21:543
2577133991512,1cyclictest0-21swapper/323:02:513
2577133991512,1cyclictest0-21swapper/320:38:523
2577133991512,1cyclictest0-21swapper/319:21:043
2577133991512,1cyclictest0-21swapper/319:16:383
2577133991512,1cyclictest0-21swapper/300:02:473
257713399151,10cyclictest2979605-21sleep00:20:003
301798140,3rtkit-daemon2713750-21wc20:50:182
28504842141,4sleep12850517-21apt-get22:40:121
257713399142,8cyclictest2934613-21taskset23:45:113
2577133991413,0cyclictest0-21swapper/322:32:363
2577133991412,1cyclictest0-21swapper/321:45:463
2577133991412,1cyclictest0-21swapper/319:32:133
2577133991412,0cyclictest0-21swapper/322:28:193
2577133991412,0cyclictest0-21swapper/320:55:233
2577133991411,1cyclictest0-21swapper/322:06:593
2577133991411,1cyclictest0-21swapper/321:20:273
2577133991411,1cyclictest0-21swapper/321:10:013
2577133991411,1cyclictest0-21swapper/320:50:443
2577133991411,1cyclictest0-21swapper/319:55:003
2577133991411,1cyclictest0-21swapper/319:40:303
257713399141,10cyclictest2966742-21taskset00:10:123
257713399141,10cyclictest2889017-21turbostat23:10:003
257713399141,10cyclictest2798698-21chrt21:58:213
257713399141,10cyclictest2773230-21chrt21:40:103
257713399141,10cyclictest2697474-21taskset20:40:173
257713399141,10cyclictest2676106-21chrt20:22:223
257713399141,10cyclictest2577490-21taskset19:10:103
2577133991410,1cyclictest0-21swapper/320:05:103
301798130,8rtkit-daemon2598505-21munin-run19:24:592
257713399131,9cyclictest2953585-21turbostat00:00:003
257713399131,9cyclictest2875529-21taskset22:55:273
257713399131,9cyclictest2669506-21taskset20:15:373
257713399131,9cyclictest2604594-21chrt19:26:053
2577133991312,0cyclictest0-21swapper/322:43:073
2577133991312,0cyclictest0-21swapper/322:17:083
2577133991311,1cyclictest0-21swapper/323:32:233
2577133991311,1cyclictest0-21swapper/321:33:433
2577133991311,1cyclictest0-21swapper/300:39:073
257713399131,10cyclictest2727785-21taskset21:05:003
2577133991310,1cyclictest0-21swapper/323:50:523
2577133991310,1cyclictest0-21swapper/300:20:243
257713399130,9cyclictest2811989-21taskset22:10:123
257713399130,10cyclictest2928105-21taskset23:40:023
26636902121,3sleep22663723-21apt-get20:15:132
257713399129,1cyclictest0-21swapper/300:30:243
257713399121,9cyclictest2662981-21taskset20:12:313
257713399121,8cyclictest2682993-21taskset20:30:033
257713399121,8cyclictest2682993-21taskset20:30:033
257713399121,8cyclictest2638159-21chrt19:55:133
257713399120,9cyclictest2986278-21taskset00:25:063
257713399120,9cyclictest2649985-21taskset20:00:303
257713399120,10cyclictest2617630-21chrt19:35:383
301798110,4rtkit-daemon636-21snmpd21:56:432
29073712110,7sleep22907420-21ntp_states23:20:232
28633242112,5sleep02863346-21apt-get22:50:110
301798100,6rtkit-daemon2643737-21runrttasks19:57:392
301798100,5rtkit-daemon2810853-21ntp_states22:05:212
29414092101,6sleep02941712-21cut23:50:140
28377702101,6sleep02837834-21fschecks_count22:30:130
28367162101,6sleep02836960-21users22:25:230
27664832101,6sleep12766510-21awk21:35:001
27489592100,1sleep00-21swapper/021:20:160
26560822101,2sleep02656231-21timerwakeupswit20:05:250
26310642102,4sleep22631097-21apt-get19:50:132
25911112100,7sleep02591182-21/usr/sbin/munin19:15:210
25704382100,6sleep30-21swapper/319:05:013
30179890,6rtkit-daemon492-21gmain21:40:342
30179890,5rtkit-daemon667-21lldpd19:47:512
30179890,5rtkit-daemon492-21gmain21:27:062
30179890,5rtkit-daemon3205-21sendmail-mta23:00:431
30179890,5rtkit-daemon3205-21sendmail-mta00:25:321
30179890,5rtkit-daemon317-21systemd-udevd22:47:332
30179890,5rtkit-daemon2592086-21true19:20:002
30179890,4rtkit-daemon448-21wpa_supplicant20:11:262
30179890,4rtkit-daemon2869735-21apt-get22:55:111
30179890,4rtkit-daemon2683038-21apt-get20:30:112
30179890,4rtkit-daemon2683038-21apt-get20:30:112
30179890,4rtkit-daemon2605606-21if_eno119:30:152
30179890,3rtkit-daemon492-21gmain00:00:061
30179890,1rtkit-daemon171rcu_preempt23:15:201
30179890,1rtkit-daemon0-21swapper/219:36:352
30179890,1rtkit-daemon0-21swapper/123:31:411
30179890,1rtkit-daemon0-21swapper/100:20:431
2930260291,5sleep02932358-21kthreadcore23:40:170
2920326291,5sleep02920452-21python323:30:220
2863349291,6sleep12863312-21/usr/sbin/munin22:50:121
2818341291,4sleep12818376-21apt-get22:15:121
2814717290,1sleep02816866-21kthreadcore22:10:170
2799104291,4sleep02799137-21apt-get22:00:120
2766753291,4sleep12766775-21apt-get21:35:131
2671584291,6sleep20-21swapper/220:20:182
2612226290,6sleep02612920-21irqrtprio19:35:150
2577333291,5sleep22577489-21diskstats19:10:102
2576085291,4sleep12576149-21ntp_states19:05:161
30179880,5rtkit-daemon3205-21sendmail-mta22:20:532
30179880,4rtkit-daemon667-21lldpd23:51:541
30179880,4rtkit-daemon667-21lldpd23:13:351
30179880,4rtkit-daemon638-21ntpd23:25:111
30179880,4rtkit-daemon636-21snmpd23:44:291
30179880,4rtkit-daemon636-21snmpd21:45:162
30179880,4rtkit-daemon636-21snmpd21:34:452
30179880,4rtkit-daemon636-21snmpd21:00:082
30179880,4rtkit-daemon636-21snmpd20:36:322
30179880,4rtkit-daemon636-21snmpd20:04:372
30179880,4rtkit-daemon492-21gmain20:25:452
30179880,4rtkit-daemon448-21wpa_supplicant22:25:532
30179880,4rtkit-daemon2837889-21perl22:30:132
30179880,4rtkit-daemon2620481-21cut19:40:172
30179880,4rtkit-daemon2583607-21munin-node21:24:042
30179880,4rtkit-daemon2583607-21munin-node20:08:552
30179880,1rtkit-daemon0-21swapper/222:35:532
30179880,1rtkit-daemon0-21swapper/222:17:352
30179880,1rtkit-daemon0-21swapper/123:06:251
2993659280,5sleep02994682-21kthreadcore00:30:170
2987779281,4sleep22989680-21grep00:25:182
2953851281,4sleep22953852-21sshd00:00:102
2928114281,3sleep22928149-21apt-get23:40:112
2922248280,6sleep22923206-21cat23:35:182
2896061281,4sleep02896330-21cut23:15:160
2876574280,5sleep22877341-21sed23:00:162
2707796281,4sleep02708040-21users20:45:240
2689480281,4sleep12689632-21df_inode20:35:141
2670587281,4sleep12671583-21irqstats20:20:171
2669850281,4sleep12669880-21munin-run20:19:591
2644594281,4sleep02644733-21sed20:00:160
2598722281,3sleep22598755-21apt-get19:25:122
30179870,4rtkit-daemon492-21gmain00:15:021
30179870,4rtkit-daemon3200908-21packagekitd23:36:101
30179870,3rtkit-daemon638-21ntpd22:10:442
30179870,3rtkit-daemon636-21snmpd23:46:191
30179870,3rtkit-daemon636-21snmpd22:50:452
30179870,3rtkit-daemon636-21snmpd21:52:142
30179870,3rtkit-daemon636-21snmpd21:17:502
30179870,3rtkit-daemon636-21snmpd20:47:462
30179870,3rtkit-daemon3205-21sendmail-mta21:36:332
30179870,3rtkit-daemon2998149-21sed00:30:231
30179870,3rtkit-daemon297-21systemd-journal22:40:112
30179870,3rtkit-daemon297-21systemd-journal21:05:112
30179870,3rtkit-daemon2583607-21munin-node23:21:241
30179870,3rtkit-daemon2583607-21munin-node22:00:092
30179870,3rtkit-daemon2583607-21munin-node20:55:072
30179870,3rtkit-daemon2583607-21munin-node00:10:451
30179870,3rtkit-daemon2583607-21munin-node00:05:151
30179870,3rtkit-daemon2537698-21munin-node19:05:102
30179870,2rtkit-daemon0-21swapper/220:40:142
2979848271,3sleep02979883-21apt-get00:20:110
2954252270,4sleep02954517-21cut00:00:150
2947407270,3sleep12947439-21apt-get23:55:121
2882890271,3sleep20-21swapper/223:05:132
2856921270,3sleep12856954-21apt-get22:45:131
2850679270,4sleep02850879-21hddtemp_smartct22:40:150
2844062271,3sleep02844214-21df_inode22:35:140
2811173271,3sleep00-21swapper/022:05:250
2798823270,4sleep02798846-21latency_hist22:00:000
2792410270,4sleep12792453-21munin-run21:55:001
2781409271,3sleep02783477-21cut21:45:170
2773393271,3sleep12773593-21hddtemp_smartct21:40:141
2760650270,4sleep02760778-21irqcore21:30:140
2759630274,2sleep02708365-21kworker/0:0+kacpid21:28:030
2739714270,4sleep22739795-21sendmail_mailst21:10:222
2715303270,4sleep12715502-21hddtemp_smartct20:55:141
2708703271,3sleep02708839-21expr20:50:110
2666163270,4sleep02668328-21cut20:15:180
2643337270,4sleep02643552-21users19:55:230
2999108261,2sleep00-21swapper/000:35:140
2992264260,1sleep00-21swapper/000:28:000
2979519260,1sleep00-21swapper/000:19:190
2973406261,2sleep22973439-21apt-get00:15:122
2966164260,1sleep00-21swapper/000:06:210
2948872261,3sleep22950703-21cut23:55:162
2927664261,3sleep0636-21snmpd23:37:440
2909499261,3sleep22910445-21grep23:25:162
2894557261,2sleep00-21swapper/023:10:200
2882210260,1sleep00-21swapper/023:03:190
2785456260,1sleep10-21swapper/121:46:151
2771822260,3sleep02771951-21/usr/sbin/munin21:35:200
2739657261,3sleep12739713-21ntp_states21:10:211
2723582260,4sleep12725534-21cut21:00:191
2682504260,1sleep10-21swapper/120:26:541
2676262260,1sleep00-21swapper/020:24:240
2637242261,2sleep12637284-21apt-key19:55:001
2618252261,2sleep00-21swapper/019:40:130
2611205260,1sleep00-21swapper/019:33:520
2598341260,1sleep00-21swapper/019:23:260
2998714250,1sleep20-21swapper/200:33:472
2972887251,2sleep20-21swapper/200:14:592
2968706250,3sleep02969864-21kthreadcore00:10:180
2961314250,2sleep20-21swapper/200:05:192
2940584250,1sleep00-21swapper/023:47:450
2915940251,2sleep20-21swapper/223:30:162
2914538250,1sleep00-21swapper/023:27:550
2901466250,2sleep20-21swapper/223:15:262
2889246250,2sleep20-21swapper/223:10:102
2875845250,1sleep00-21swapper/022:59:300
2875560250,1sleep20-21swapper/222:55:512
2843682250,1sleep10-21swapper/122:33:551
2830662250,1sleep00-21swapper/022:21:440
2824783250,2sleep10-21swapper/122:20:121
2811539250,1sleep10-21swapper/122:07:581
2800403251,2sleep10-21swapper/122:00:171
2786263250,2sleep00-21swapper/021:50:100
2774914250,2sleep00-21swapper/021:40:170
2759058251,1sleep10-21swapper/121:25:231
2746802250,1sleep10-21swapper/121:18:091
2740490250,1sleep00-21swapper/021:14:120
2734038250,1sleep00-21swapper/021:08:470
2728250251,2sleep10-21swapper/121:05:161
2721104250,1sleep00-21swapper/020:57:220
2714735250,1sleep10-21swapper/120:53:331
2708359250,1sleep10-21swapper/120:49:211
2696249250,2sleep12696508-21irqcore20:40:161
2689444251,1sleep00-21swapper/020:35:140
2688871250,3sleep00-21swapper/020:30:570
2688871250,3sleep00-21swapper/020:30:560
2682135252,1sleep00-21swapper/020:25:240
2650575250,1sleep10-21swapper/120:05:021
2650245250,1sleep10-21swapper/120:03:531
2639733251,1sleep10-21swapper/119:55:161
2630641250,1sleep10-21swapper/119:47:511
2617717250,1sleep10-21swapper/119:36:441
2604176250,2sleep12604285-21find19:25:231
2583464250,3sleep1321rcuc/119:15:021
2583242250,1sleep10-21swapper/119:12:091
2576374250,1sleep00-21swapper/019:05:190
2982013241,1sleep2421rcuc/200:20:172
2953542240,3sleep00-21swapper/023:59:440
2942716240,2sleep20-21swapper/223:50:162
2934587240,1sleep20-21swapper/223:45:112
2888169240,2sleep02888277-21mailstats23:05:220
2850183240,1sleep10-21swapper/122:39:441
2824192240,3sleep00-21swapper/022:16:030
2795125240,2sleep10-21swapper/121:55:151
2747417240,2sleep10-21swapper/121:20:141
2742264240,2sleep00-21swapper/021:15:170
2721581240,3sleep00-21swapper/021:00:100
2701675240,3sleep00-21swapper/020:40:590
2658085240,2sleep12659339-21kthreadcore20:10:151
2618287240,1sleep10-21swapper/119:40:141
2604660240,3sleep00-21swapper/019:26:550
2595534240,2sleep12597413-21sed19:20:181
2577694240,2sleep00-21swapper/019:10:110
2999273230,2sleep22999474-21smartctl00:35:152
2907500230,1sleep00-21swapper/023:20:250
2856908230,2sleep00-21swapper/022:45:030
2831208230,1sleep10-21swapper/122:25:121
2816869230,1sleep10-21swapper/122:10:181
2657022230,1sleep00-21swapper/020:10:100
2631255230,1sleep00-21swapper/019:50:140
2626867230,1sleep02629012-21cut19:45:180
2682991220,1sleep10-21swapper/120:30:021
2682991220,1sleep10-21swapper/120:30:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional