You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-26 - 05:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Wed Aug 26, 2026 00:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
128452240164,25sleep00-21swapper/019:05:480
129052234162,24sleep30-21swapper/319:06:373
131162209158,38sleep20-21swapper/219:09:202
129702203167,24sleep10-21swapper/119:07:261
2818721660,3sleep11325999cyclictest22:08:091
1190721530,4sleep31327199cyclictest21:30:193
13265998178,2cyclictest0-21swapper/221:31:452
13265997871,2cyclictest0-21swapper/221:35:402
13265996763,3cyclictest0-21swapper/223:45:212
13265995047,2cyclictest0-21swapper/200:30:332
48152470,1sleep20-21swapper/221:15:182
76562420,2sleep20-21swapper/221:20:242
13271993521,8cyclictest0-21swapper/322:51:453
234752340,4sleep01325299cyclictest21:55:240
13252993310,11cyclictest8566-21awk21:25:010
1327199321,24cyclictest0-21swapper/300:13:053
1327199321,22cyclictest0-21swapper/320:35:053
1325999321,18cyclictest0-21swapper/120:20:511
13271993115,4cyclictest0-21swapper/320:20:023
1327199311,24cyclictest0-21swapper/319:20:013
1327199311,23cyclictest0-21swapper/321:28:093
1327199311,18cyclictest0-21swapper/320:43:373
1327199311,14cyclictest0-21swapper/320:50:333
1327199311,13cyclictest0-21swapper/323:58:053
1327199310,11cyclictest0-21swapper/320:45:293
1325999311,23cyclictest0-21swapper/121:06:001
1325999311,11cyclictest0-21swapper/119:57:111
1325999310,23cyclictest0-21swapper/100:21:591
13271993017,7cyclictest32083-21cat22:15:263
1327199301,18cyclictest0-21swapper/319:57:103
1327199301,14cyclictest0-21swapper/320:56:373
1327199301,14cyclictest0-21swapper/319:20:213
1327199301,13cyclictest0-21swapper/322:35:453
1327199301,13cyclictest0-21swapper/322:21:423
1327199301,13cyclictest0-21swapper/322:21:423
1327199301,12cyclictest0-21swapper/320:33:453
1327199300,12cyclictest0-21swapper/322:07:583
13265993026,3cyclictest0-21swapper/222:27:432
13259993011,3cyclictest0-21swapper/122:26:271
1325999301,12cyclictest0-21swapper/120:04:311
1325999301,11cyclictest0-21swapper/119:18:481
1325299301,14cyclictest14539-21munin-run20:25:020
1327199291,13cyclictest0-21swapper/323:52:133
1327199291,11cyclictest0-21swapper/322:33:373
1327199290,13cyclictest0-21swapper/300:38:423
1326599291,13cyclictest0-21swapper/219:58:352
1325999291,5cyclictest2191-21/usr/sbin/munin21:10:191
1325999291,10cyclictest0-21swapper/123:49:521
1325999290,22cyclictest0-21swapper/122:35:351
1325999290,22cyclictest0-21swapper/121:46:591
1325299296,11cyclictest18322-21munin-run19:20:020
274312280,3sleep01325299cyclictest00:35:010
1327199281,13cyclictest0-21swapper/300:34:343
1327199281,12cyclictest0-21swapper/321:39:333
1327199281,12cyclictest0-21swapper/320:14:423
1327199280,13cyclictest0-21swapper/323:43:413
13259992810,3cyclictest0-21swapper/119:33:271
1325999280,16cyclictest0-21swapper/120:27:191
1325299281,12cyclictest21142-21basename20:40:020
13252992810,15cyclictest18610-21munin-run00:15:010
1327199271,10cyclictest0-21swapper/300:09:333
1325999271,9cyclictest0-21swapper/123:37:521
1325999271,9cyclictest0-21swapper/100:15:391
1325999271,8cyclictest0-21swapper/121:04:361
1325999271,8cyclictest0-21swapper/100:01:321
1325999271,11cyclictest0-21swapper/123:40:281
1325999271,11cyclictest0-21swapper/122:00:391
1325999270,18cyclictest0-21swapper/120:47:231
1325999270,16cyclictest0-21swapper/121:15:361
1325999270,10cyclictest0-21swapper/123:25:521
1325299271,12cyclictest23017-21basename00:25:000
1325299270,12cyclictest16426-21awk00:10:010
1327199269,3cyclictest0-21swapper/321:06:493
1327199269,3cyclictest0-21swapper/300:17:013
1327199268,3cyclictest0-21swapper/319:30:103
1327199261,9cyclictest0-21swapper/321:15:293
13271992615,3cyclictest0-21swapper/320:20:573
1327199261,23cyclictest0-21swapper/322:00:203
1327199261,18cyclictest0-21swapper/300:21:063
13271992611,3cyclictest0-21swapper/323:37:293
1327199261,12cyclictest0-21swapper/322:49:153
1327199261,12cyclictest0-21swapper/321:56:583
1327199261,12cyclictest0-21swapper/319:53:423
1327199261,11cyclictest0-21swapper/323:34:293
1327199260,13cyclictest0-21swapper/322:11:423
1327199260,12cyclictest0-21swapper/323:07:053
1327199260,12cyclictest0-21swapper/320:04:423
1327199260,10cyclictest0-21swapper/323:46:213
1325999268,7cyclictest0-21swapper/123:00:071
1325999268,3cyclictest0-21swapper/120:05:391
1325999261,9cyclictest0-21swapper/122:34:231
1325999261,9cyclictest0-21swapper/119:46:151
1325999261,8cyclictest0-21swapper/120:42:121
1325999261,11cyclictest0-21swapper/122:45:551
1325999261,11cyclictest0-21swapper/119:52:031
1325999261,11cyclictest0-21swapper/119:22:051
1325999260,12cyclictest0-21swapper/100:37:071
1325999260,11cyclictest0-21swapper/123:19:561
1325999260,11cyclictest0-21swapper/121:57:191
1325999260,10cyclictest0-21swapper/120:33:431
1325299261,13cyclictest1965-21munin-run21:10:010
1325299260,13cyclictest11385-21idleruntime-cro22:45:020
1327199257,3cyclictest0-21swapper/320:25:173
1327199251,9cyclictest0-21swapper/323:25:473
1327199251,12cyclictest0-21swapper/319:26:063
1327199251,11cyclictest0-21swapper/323:10:013
13271992510,4cyclictest0-21swapper/319:40:303
1327199250,9cyclictest0-21swapper/323:23:293
1325999251,9cyclictest0-21swapper/122:40:071
1325999251,10cyclictest0-21swapper/119:29:411
1325999250,9cyclictest0-21swapper/123:13:431
1325999250,9cyclictest0-21swapper/119:12:151
1325999250,10cyclictest0-21swapper/123:50:231
1325999250,10cyclictest0-21swapper/121:41:311
1325999250,10cyclictest0-21swapper/100:31:391
1327199249,3cyclictest0-21swapper/319:37:103
1327199248,3cyclictest0-21swapper/321:04:333
1327199246,3cyclictest0-21swapper/321:21:013
1327199241,12cyclictest0-21swapper/300:02:133
1327199241,10cyclictest0-21swapper/321:53:103
1327199240,13cyclictest0-21swapper/321:48:383
1327199240,11cyclictest0-21swapper/323:15:053
1327199240,11cyclictest0-21swapper/322:57:413
1327199240,11cyclictest0-21swapper/319:10:053
1325999248,3cyclictest31346-21perl22:15:161
1325999247,3cyclictest0-21swapper/119:35:411
1325999241,9cyclictest0-21swapper/120:37:351
1325999241,11cyclictest1961-21diskmemload21:35:051
1325999241,11cyclictest10761-21awk21:30:011
1325999241,10cyclictest0-21swapper/120:17:211
1325999241,10cyclictest0-21swapper/120:10:091
1325999241,10cyclictest0-21swapper/119:40:371
1325999240,8cyclictest0-21swapper/120:51:521
1325999240,8cyclictest0-21swapper/100:10:511
1325999240,11cyclictest0-21swapper/123:07:191
1325999240,10cyclictest0-21swapper/122:55:311
1325299247,15cyclictest6371-21perf21:20:010
1327199239,3cyclictest0-21swapper/300:27:543
1327199238,3cyclictest0-21swapper/323:01:493
1327199236,3cyclictest0-21swapper/322:26:053
1327199236,3cyclictest0-21swapper/321:42:183
1327199234,3cyclictest0-21swapper/322:42:333
1327199231,12cyclictest0-21swapper/321:11:493
1325999235,2cyclictest0-21swapper/123:59:241
1325999231,8cyclictest0-21swapper/121:20:121
1325999231,11cyclictest0-21swapper/122:11:431
1325999231,10cyclictest0-21swapper/120:55:201
1325999230,9cyclictest0-21swapper/123:21:521
1325299231,11cyclictest29594-21munin-run00:40:010
1327199223,9cyclictest0-21swapper/319:45:543
1327199221,20cyclictest0-21swapper/320:06:503
1326599226,7cyclictest32-21ksoftirqd/221:10:012
1326599221,9cyclictest0-21swapper/222:51:112
1326599221,8cyclictest0-21swapper/200:02:572
1326599221,4cyclictest0-21swapper/221:49:442
1326599221,15cyclictest0-21swapper/222:20:272
1326599221,15cyclictest0-21swapper/222:20:272
1325999228,3cyclictest0-21swapper/121:31:151
1325999221,9cyclictest0-21swapper/122:22:531
1325999221,9cyclictest0-21swapper/122:22:531
1325999221,7cyclictest0-21swapper/121:54:391
1325999221,6cyclictest0-21swapper/123:34:521
1325999221,13cyclictest0-21swapper/100:26:021
1325999221,10cyclictest0-21swapper/122:53:231
13265992115,3cyclictest27623-21sed23:20:152
1325999211,7cyclictest0-21swapper/100:05:211
1325299211,9cyclictest11357-21grep20:15:210
1325299211,4cyclictest0-21swapper/023:58:250
13252992114,4cyclictest11327-21diskstats21:30:140
1325299211,11cyclictest20397-21/usr/sbin/munin23:05:190
1325299211,10cyclictest31467-21cpuspeed_turbos23:30:140
1326599202,8cyclictest11946-21/usr/sbin/munin22:45:132
1326599201,6cyclictest0-21swapper/222:33:272
13265992013,4cyclictest27789-21tr00:35:132
13265992011,5cyclictest32043-21awk22:15:252
1325299203,14cyclictest13463-21fgrep00:00:240
13252992013,4cyclictest26967-21gltestperf22:05:160
193972190,2sleep00-21swapper/021:48:150
1326599199,3cyclictest24575-21awk23:15:002
1326599192,15cyclictest0-21swapper/200:13:122
1326599191,4cyclictest0-21swapper/221:14:272
1326599191,12cyclictest0-21swapper/222:02:532
13265991910,3cyclictest87550irq/128-enp1s0-23:01:092
1325299198,3cyclictest25043-21sshd00:28:100
1325299194,9cyclictest28771-21munin-node19:14:240
1325299194,3cyclictest26764-21sshd19:35:460
1325299194,14cyclictest0-21swapper/023:20:010
1325299191,7cyclictest0-21swapper/022:49:190
13252991914,3cyclictest20809-21munin-run00:20:000
13252991912,4cyclictest17525-21grep22:55:270
1325299191,12cyclictest0-21swapper/019:45:250
1325299191,11cyclictest29837-21sshd20:59:100
1325299191,10cyclictest808-21snmpd23:10:110
1325299191,10cyclictest30767-21/usr/sbin/munin22:15:250
1325299190,2cyclictest0-21swapper/021:40:030
106772191,2sleep20-21swapper/221:29:152
1326599188,3cyclictest0-21swapper/223:30:172
1326599183,7cyclictest32-21ksoftirqd/220:55:122
1326599183,13cyclictest9358-21grep23:50:272
1326599181,7cyclictest29548-21egrep19:45:132
1326599181,7cyclictest0-21swapper/200:20:282
1326599181,6cyclictest0-21swapper/223:58:272
1326599181,5cyclictest0-21swapper/222:08:492
1326599181,5cyclictest0-21swapper/220:29:432
13265991814,3cyclictest0-21swapper/221:40:342
13265991814,2cyclictest32-21ksoftirqd/222:00:012
13265991811,4cyclictest10131-21latency_hist20:15:012
1325299189,5cyclictest0-21swapper/020:04:560
1325299187,6cyclictest0-21swapper/023:54:560
1325299187,3cyclictest810-21sshd22:20:100
1325299187,3cyclictest810-21sshd22:20:100
1325299187,3cyclictest5005-21/usr/sbin/munin22:30:170
1325299187,3cyclictest14766-21/usr/sbin/munin20:25:130
13252991813,3cyclictest4815-21cut22:30:020
13252991812,3cyclictest24165-21gltestperf20:45:180
1325299181,10cyclictest30181-21cut23:25:200
1325299181,10cyclictest0-21swapper/021:03:320
56562170,2sleep20-21swapper/220:04:062
207862171,2sleep20-21swapper/220:35:492
201172170,2sleep00-21swapper/023:04:290
1326599172,3cyclictest0-21swapper/219:20:542
1326599171,8cyclictest0-21swapper/200:28:162
1326599171,7cyclictest4523-21awk23:40:222
1326599171,6cyclictest0-21swapper/200:15:142
1326599171,4cyclictest23648-21grep20:45:132
13265991713,3cyclictest0-21swapper/223:18:192
13265991713,3cyclictest0-21swapper/220:23:392
13265991712,3cyclictest28573-21/usr/sbin/munin22:10:192
13265991712,3cyclictest11741-21wc20:15:252
1326599171,14cyclictest0-21swapper/222:40:092
1326599171,14cyclictest0-21swapper/219:30:072
13265991711,3cyclictest25868-21tr20:50:132
1326599171,10cyclictest0-21swapper/223:29:472
1325299179,1cyclictest0-21swapper/019:20:140
1325299175,3cyclictest87550irq/128-enp1s0-23:40:190
1325299174,4cyclictest23254-21cat19:30:140
1325299171,7cyclictest0-21swapper/021:35:170
1325299171,6cyclictest21658-21memory19:25:190
1325299171,6cyclictest0-21swapper/023:22:110
1325299171,5cyclictest0-21swapper/022:37:390
1325299171,5cyclictest0-21swapper/020:50:140
13252991712,3cyclictest5617-21/usr/sbin/munin23:45:170
13252991712,3cyclictest21738-21idleruntime-cro21:55:010
1325299171,15cyclictest0-21swapper/020:31:150
13252991711,3cyclictest8264-21grep20:10:140
1325299171,10cyclictest0-21swapper/022:50:160
1326599169,5cyclictest15014-21unixbench_singl19:10:252
1326599163,4cyclictest21363-21users21:50:272
1326599163,2cyclictest0-21swapper/220:40:122
1326599162,4cyclictest30-21rcuc/219:35:172
1326599161,7cyclictest0-21swapper/223:05:152
1326599161,6cyclictest0-21swapper/222:56:502
13265991612,3cyclictest0-21swapper/221:00:462
13265991612,3cyclictest0-21swapper/220:09:092
13265991612,3cyclictest0-21swapper/219:25:592
13265991611,3cyclictest7200-21/usr/sbin/munin22:35:112
13265991611,3cyclictest31792-21cpuspeed_turbos19:50:122
13265991611,3cyclictest15889-21if_err_enp1s019:15:032
13265991611,3cyclictest1565-21diskstats23:35:142
13265991610,4cyclictest0-21swapper/200:05:162
1325299161,6cyclictest28245-21load19:40:190
1325299161,5cyclictest3325-21grep21:10:230
1325299161,4cyclictest1880-21gltestperf23:35:150
1325299161,4cyclictest0-21swapper/022:00:300
1325299161,14cyclictest0-21swapper/019:51:360
13252991610,3cyclictest21495-21grep20:40:130
1325299160,9cyclictest0-21swapper/022:10:270
1326599159,3cyclictest17106-21cut20:30:132
1326599151,5cyclictest27334-21/usr/sbin/munin19:40:262
1325299159,3cyclictest1713-21grep19:55:120
1325299151,2cyclictest0-21swapper/021:25:040
13252991510,3cyclictest7275-21wc20:05:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional