You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-13 - 12:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Sat Jun 13, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
56372240164,25sleep00-21swapper/019:07:140
58222204167,24sleep20-21swapper/219:09:372
58162204173,20sleep10-21swapper/119:09:321
55302202165,25sleep30-21swapper/319:05:523
2127321560,5sleep0593599cyclictest00:35:180
4552500,1sleep20-21swapper/221:20:232
45902490,2sleep00-21swapper/022:45:130
209002440,3sleep10-21swapper/123:20:221
5947994311,2cyclictest21664-21awk23:25:012
5947993712,3cyclictest6551-21hwlatdetect21:35:162
5947993712,3cyclictest3043-21/usr/sbin/munin20:15:122
594799361,21cyclictest0-21swapper/220:51:512
5947993611,3cyclictest14060-21df00:20:132
595299351,20cyclictest0-21swapper/322:28:533
594799349,3cyclictest30799-21awk20:05:012
5947993413,3cyclictest16856-21/usr/sbin/munin22:00:242
5947993411,4cyclictest1768-21smart_sda23:50:232
5947993411,3cyclictest10263-21/usr/sbin/munin21:45:162
593599346,13cyclictest6916-21idleruntime-cro00:05:010
593599346,13cyclictest29845-21basename22:30:010
595299331,13cyclictest22516-21taskset20:59:053
595299331,13cyclictest0-21swapper/322:44:253
594799333,4cyclictest29779-21diskstats21:15:142
594799333,4cyclictest13689-21gltestperf23:05:162
5947993311,3cyclictest24409-21/usr/sbin/munin19:50:222
595299321,20cyclictest0-21swapper/323:04:123
595299321,20cyclictest0-21swapper/321:21:443
595299321,14cyclictest0-21swapper/300:03:453
594799325,3cyclictest87350irq/126-enp1s0-19:55:142
594799322,6cyclictest8564-1kworker/u9:122:26:042
594799321,5cyclictest31113-21ntp_states22:30:172
5947993213,4cyclictest429-21sed22:35:122
594799321,14cyclictest1578-21grep21:25:092
5947993210,3cyclictest31725-21ntpq23:45:192
5947993210,3cyclictest31010-21/usr/sbin/munin20:05:182
593599324,13cyclictest21037-21munin-run22:10:010
595299311,20cyclictest0-21swapper/319:19:403
595299311,19cyclictest0-21swapper/320:21:133
595299311,19cyclictest0-21swapper/300:12:573
595299311,15cyclictest0-21swapper/322:03:053
595299311,15cyclictest0-21swapper/300:31:093
595299311,14cyclictest0-21swapper/300:19:133
595299311,13cyclictest0-21swapper/321:54:373
595299310,13cyclictest0-21swapper/300:08:013
594799316,3cyclictest87350irq/126-enp1s0-19:45:142
594799314,3cyclictest827-21runrttasks20:28:312
5947993118,3cyclictest0-21swapper/200:31:272
594799311,18cyclictest0-21swapper/222:09:112
5947993111,3cyclictest2153-21switchtime20:10:232
5947993110,4cyclictest0-21swapper/200:05:292
5947993110,3cyclictest29701-21sendmail_mailqu23:40:232
595299301,19cyclictest0-21swapper/323:13:573
595299301,19cyclictest0-21swapper/323:13:573
595299301,14cyclictest0-21swapper/322:30:493
595299301,13cyclictest0-21swapper/319:37:213
595299301,13cyclictest0-21swapper/319:37:213
595299301,12cyclictest0-21swapper/323:29:173
595299301,12cyclictest0-21swapper/322:58:493
595299301,12cyclictest0-21swapper/300:27:293
5952993010,3cyclictest0-21swapper/320:26:293
594799304,3cyclictest87350irq/126-enp1s0-22:10:122
594799303,4cyclictest3236-21kworker/u8:121:30:252
594799303,4cyclictest12517-21chrt23:00:482
594799302,6cyclictest11551-21egrep00:15:132
594799301,7cyclictest23456-21/usr/sbin/munin22:15:212
594799301,6cyclictest0-21swapper/223:13:042
594799301,6cyclictest0-21swapper/223:13:042
594799301,5cyclictest0-21swapper/222:45:132
594799301,5cyclictest0-21swapper/200:36:572
594799301,2cyclictest87350irq/126-enp1s0-00:01:072
5947993010,3cyclictest9446-21cpuspeed_turbos00:10:122
595299291,20cyclictest0-21swapper/322:36:453
595299291,20cyclictest0-21swapper/322:23:213
595299291,17cyclictest0-21swapper/323:06:253
595299291,13cyclictest0-21swapper/323:24:493
595299291,13cyclictest0-21swapper/319:51:253
595299291,12cyclictest0-21swapper/323:48:533
595299291,11cyclictest0-21swapper/322:45:413
595299291,11cyclictest0-21swapper/320:35:353
595299291,10cyclictest0-21swapper/319:27:473
595299290,18cyclictest0-21swapper/323:37:293
594799291,6cyclictest0-21swapper/222:43:242
5947992911,2cyclictest14252-21expr20:40:122
5947992910,3cyclictest21677-21ntpq20:55:212
593599293,13cyclictest7841-21munin-run21:40:010
595299282,13cyclictest25431-21munin-run22:20:013
595299281,15cyclictest0-21swapper/323:43:013
595299281,13cyclictest0-21swapper/322:12:253
595299281,13cyclictest0-21swapper/321:38:013
595299281,13cyclictest0-21swapper/320:12:053
595299281,12cyclictest0-21swapper/323:51:533
595299281,12cyclictest0-21swapper/323:30:573
595299281,12cyclictest0-21swapper/321:49:053
595299281,12cyclictest0-21swapper/319:12:113
595299281,11cyclictest0-21swapper/321:40:333
594799285,4cyclictest27003-21diskmemload00:26:072
594799281,5cyclictest8007-21unixbench_multi22:50:252
594799281,4cyclictest13496-21ntpq21:50:212
5947992811,3cyclictest16601-21needreboot19:30:192
5947992810,3cyclictest9109-21cat19:15:112
5935992810,16cyclictest10052-21idleruntime-cro21:45:010
595299271,13cyclictest0-21swapper/321:29:203
595299271,12cyclictest0-21swapper/322:05:573
595299271,11cyclictest0-21swapper/321:31:093
595299271,11cyclictest0-21swapper/321:07:213
595299271,10cyclictest0-21swapper/320:45:253
595299271,10cyclictest0-21swapper/320:05:053
594799276,3cyclictest0-21swapper/219:35:152
594799276,3cyclictest0-21swapper/219:35:152
594799271,5cyclictest0-21swapper/221:14:472
594799271,4cyclictest13195-21latency_hist19:25:012
594799271,3cyclictest0-21swapper/223:25:232
593599270,12cyclictest27022-21idleruntime-cro21:10:010
5952992616,3cyclictest0-21swapper/323:18:173
595299261,12cyclictest0-21swapper/321:13:083
595299261,11cyclictest0-21swapper/322:52:253
595299261,11cyclictest0-21swapper/300:21:173
595299261,10cyclictest0-21swapper/320:40:293
595299261,10cyclictest0-21swapper/320:15:053
595299260,10cyclictest0-21swapper/321:17:063
594799262,4cyclictest21400-21timerandwakeup19:40:232
594799261,6cyclictest0-21swapper/223:33:272
594799261,5cyclictest27085-21cat21:10:022
594799261,4cyclictest808-21snmpd20:20:292
594799261,3cyclictest27003-21diskmemload22:58:232
5947992610,3cyclictest2942-21expr23:55:132
594799260,6cyclictest0-21swapper/222:22:362
594299260,24cyclictest0-21swapper/119:59:551
593599267,16cyclictest20406-21munin-run20:55:010
593599263,11cyclictest13826-21perf20:40:010
5952992512,3cyclictest0-21swapper/300:35:053
595299251,10cyclictest0-21swapper/323:56:293
594799251,3cyclictest808-21snmpd20:30:322
5947992512,2cyclictest0-21swapper/223:36:582
594799251,11cyclictest0-21swapper/220:39:392
5947992510,9cyclictest0-21swapper/223:16:472
594299257,4cyclictest1-21systemd00:30:001
594299251,3cyclictest9111-21munin-run00:10:011
5942992513,5cyclictest31759-21systemd-journal22:31:471
593599256,16cyclictest7226-21munin-run20:25:010
5935992516,3cyclictest28195-21users19:55:260
593599251,12cyclictest24188-21munin-run19:50:010
593599251,11cyclictest22488-21kworker/u8:221:10:260
595299241,9cyclictest0-21swapper/321:00:573
595299241,9cyclictest0-21swapper/319:23:253
595299241,11cyclictest29873-21ntp_states20:00:213
595299240,9cyclictest0-21swapper/320:30:493
594799248,3cyclictest6375-21cstates19:10:102
594299243,6cyclictest31346-40dnf21:19:321
594299242,19cyclictest909-21sssd_nss21:35:211
594299241,10cyclictest0-21swapper/121:59:541
593599247,2cyclictest7281-21grep19:10:210
593599241,6cyclictest0-21swapper/022:50:060
593599241,16cyclictest0-21swapper/000:08:560
593599241,15cyclictest0-21swapper/023:27:590
14982240,4sleep10-21swapper/120:10:151
595299231,8cyclictest0-21swapper/321:55:273
595299231,14cyclictest0-21swapper/319:40:453
595299231,11cyclictest0-21swapper/319:57:493
594799235,3cyclictest0-21swapper/221:44:152
594299236,5cyclictest827-21runrttasks00:13:301
594299232,18cyclictest22548-21df19:45:131
594299231,19cyclictest24750-21df_inode19:50:131
594299231,13cyclictest0-21swapper/122:00:061
594299231,10cyclictest4004-21df21:30:131
5942992310,11cyclictest2266-21sshd23:52:291
593599232,9cyclictest15702-21munin-run00:25:010
593599231,9cyclictest32154-21tr20:05:220
593599231,6cyclictest0-21swapper/023:46:340
593599231,6cyclictest0-21swapper/022:44:250
593599231,6cyclictest0-21swapper/000:13:340
593599231,10cyclictest10683-21latency_hist23:00:010
593599231,10cyclictest0-21swapper/020:29:270
593599230,10cyclictest0-21swapper/023:10:190
593599230,10cyclictest0-21swapper/023:10:190
595299221,9cyclictest0-21swapper/319:33:253
594799226,3cyclictest0-21swapper/221:55:592
594799221,14cyclictest0-21swapper/219:26:112
594299223,6cyclictest15340-40dnf00:20:361
594299222,18cyclictest0-21swapper/122:21:091
594299222,18cyclictest0-21swapper/120:56:051
594299221,4cyclictest20413-21perf20:55:011
594299221,3cyclictest10455-21interrupts20:30:171
594299221,3cyclictest10028-21timerandwakeup22:55:241
593599228,3cyclictest14933-21sshd19:25:250
593599227,2cyclictest21524-21sed19:40:230
593599221,7cyclictest29794-21strings20:00:190
593599221,6cyclictest0-21swapper/000:29:480
593599221,10cyclictest19370-21grep19:35:250
593599221,10cyclictest19370-21grep19:35:250
593599221,10cyclictest0-21swapper/021:15:030
593599220,9cyclictest0-21swapper/022:00:200
131272220,5sleep121-21rcuc/121:50:161
595299211,9cyclictest0-21swapper/320:54:173
595299211,13cyclictest0-21swapper/319:45:253
594799216,3cyclictest0-21swapper/220:47:232
5942992115,3cyclictest28818-21diskstats23:40:141
5942992115,1cyclictest0-21swapper/119:10:401
594299211,3cyclictest32073-21ntp_states20:05:211
594299211,2cyclictest6839-21chrt00:04:241
594299211,18cyclictest0-21swapper/121:45:171
594299211,11cyclictest0-21swapper/120:01:511
594299210,19cyclictest0-21swapper/121:00:211
593599218,2cyclictest14085-21head23:05:190
593599218,1cyclictest0-21swapper/023:20:180
593599217,9cyclictest0-21swapper/020:32:580
593599212,9cyclictest19185-21ntp_states00:30:210
593599211,9cyclictest0-21swapper/022:15:110
593599211,7cyclictest25991-21df22:20:130
593599211,5cyclictest0-21swapper/022:31:030
593599211,5cyclictest0-21swapper/022:11:500
593599211,17cyclictest0-21swapper/022:35:150
593599211,12cyclictest16224-21grep19:30:150
593599211,10cyclictest0-21swapper/023:31:510
593599210,8cyclictest0-21swapper/021:49:120
593599210,6cyclictest0-21swapper/023:35:470
594299206,3cyclictest5355-21ntp_states22:45:221
594299203,6cyclictest0-21swapper/123:31:361
594299202,3cyclictest0-21swapper/123:46:281
594299202,3cyclictest0-21swapper/123:37:461
594299202,3cyclictest0-21swapper/119:44:331
594299202,17cyclictest0-21swapper/120:28:311
594299202,16cyclictest0-21swapper/123:02:561
594299201,8cyclictest27003-21diskmemload23:08:111
594299201,7cyclictest8979-21memory21:40:191
5942992016,3cyclictest0-21swapper/100:16:311
594299201,4cyclictest12829-21strings20:35:181
5942992014,3cyclictest15589-21unixbench_singl20:40:251
5942992010,3cyclictest27046-21cat21:10:011
594299200,18cyclictest0-21swapper/120:18:211
593599209,3cyclictest14303-21sshd21:53:280
593599204,8cyclictest0-21swapper/020:45:040
593599201,9cyclictest0-21swapper/023:42:270
593599201,9cyclictest0-21swapper/023:04:160
593599201,9cyclictest0-21swapper/021:27:520
593599200,6cyclictest0-21swapper/020:10:220
279042201,3sleep127954-21if_enp1s021:10:161
594799193,4cyclictest0-21swapper/221:02:032
594299197,10cyclictest23-21ksoftirqd/119:30:171
594299195,12cyclictest21531-21cstates22:10:111
594299192,6cyclictest0-21swapper/100:33:161
594299192,3cyclictest0-21swapper/119:21:141
594299192,16cyclictest0-21swapper/121:25:561
594299192,16cyclictest0-21swapper/121:22:161
594299192,16cyclictest0-21swapper/120:23:471
594299192,16cyclictest0-21swapper/100:35:091
594299191,6cyclictest0-21swapper/120:46:041
5942991915,3cyclictest0-21swapper/122:18:131
5942991915,3cyclictest0-21swapper/119:26:591
594299191,4cyclictest1-21systemd19:15:021
5942991913,3cyclictest1395-21grep22:35:231
5942991912,4cyclictest2733-21gltestperf22:40:151
594299191,17cyclictest0-21swapper/123:15:391
594299191,17cyclictest0-21swapper/123:14:031
594299191,17cyclictest0-21swapper/123:14:031
594299191,17cyclictest0-21swapper/119:39:311
594299191,17cyclictest0-21swapper/119:39:311
594299191,16cyclictest0-21swapper/123:25:301
594299191,16cyclictest0-21swapper/122:51:041
593599195,3cyclictest1936-21timerwakeupswit23:50:240
593599192,9cyclictest0-21swapper/023:58:140
593599191,8cyclictest27003-21diskmemload00:15:280
593599191,6cyclictest0-21swapper/021:30:550
593599191,6cyclictest0-21swapper/020:40:230
593599191,15cyclictest32230-21tail21:20:150
593599191,12cyclictest0-21swapper/023:15:110
594299181,5cyclictest2701-21/usr/sbin/munin23:55:151
594299181,1cyclictest0-21swapper/122:25:021
594299181,14cyclictest19916-21latency22:05:161
593599186,2cyclictest0-21swapper/020:55:180
593599181,6cyclictest0-21swapper/021:56:050
593599181,6cyclictest0-21swapper/021:01:390
593599181,5cyclictest0-21swapper/020:15:150
593599178,1cyclictest0-21swapper/019:50:160
593599172,3cyclictest12037-21irqstats19:20:160
593599169,4cyclictest763-21dbus-daemon19:15:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional