You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-16 - 14:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sun Aug 16, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
37932240164,24sleep10-21swapper/107:06:461
36742228197,21sleep00-21swapper/007:05:280
39372203166,25sleep30-21swapper/307:08:383
37372203167,23sleep20-21swapper/207:06:032
242182710,1sleep30-21swapper/310:20:173
4352530,4sleep230-21rcuc/210:40:132
97492480,4sleep07-21ksoftirqd/009:45:250
411999376,14cyclictest8857-21munin-run11:00:020
4119993510,12cyclictest23007-21munin-run09:05:010
413899331,14cyclictest0-21swapper/311:27:313
413899331,14cyclictest0-21swapper/311:27:313
411999336,12cyclictest0-21swapper/011:15:010
413899311,10cyclictest6080-21cat10:50:253
411999312,14cyclictest574-21munin-run11:55:010
411999311,15cyclictest30829-21munin-run11:50:010
32712310,3sleep0411999cyclictest12:00:140
413899301,12cyclictest0-21swapper/310:07:143
413899301,11cyclictest0-21swapper/310:15:303
413899292,11cyclictest7943-21ntp_states10:55:213
413899292,10cyclictest11959-21hddtemp_smartct11:05:173
413899291,14cyclictest0-21swapper/308:28:103
413899291,13cyclictest0-21swapper/312:04:303
413399291,12cyclictest0-21swapper/210:30:392
4133992910,8cyclictest0-21swapper/211:10:142
412899298,17cyclictest12402-21cstates08:40:141
411999295,11cyclictest31168-21idleruntime-cro08:10:010
411999291,16cyclictest26476-21sa109:10:200
413899281,13cyclictest0-21swapper/311:01:533
4138992810,3cyclictest0-21swapper/312:20:223
413399281,11cyclictest0-21swapper/210:15:072
413399280,13cyclictest0-21swapper/208:28:072
413399280,12cyclictest0-21swapper/210:38:432
411999281,18cyclictest0-21swapper/011:20:240
411999280,18cyclictest5759-21open_files10:50:230
4138992721,3cyclictest22934-21hddtemp_smartct11:30:163
4138992721,3cyclictest22934-21hddtemp_smartct11:30:163
4138992711,3cyclictest26752-21idleruntime10:25:183
413899271,13cyclictest0-21swapper/311:20:173
413899271,13cyclictest0-21swapper/307:51:183
413899271,12cyclictest0-21swapper/309:02:533
4138992710,3cyclictest14274-21sed09:55:263
413399271,9cyclictest0-21swapper/208:43:392
413399271,10cyclictest0-21swapper/211:44:362
413399270,9cyclictest0-21swapper/209:30:322
413399270,12cyclictest0-21swapper/211:00:592
413899261,13cyclictest0-21swapper/309:05:133
413899261,13cyclictest0-21swapper/309:05:133
413899261,12cyclictest0-21swapper/308:43:453
4138992610,3cyclictest29599-21/usr/sbin/munin09:20:013
4138992610,3cyclictest0-21swapper/311:51:293
4138992610,3cyclictest0-21swapper/308:53:333
4138992610,10cyclictest0-21swapper/312:33:303
413899260,12cyclictest0-21swapper/308:58:373
4128992610,13cyclictest8274-21ntpq07:15:221
411999262,11cyclictest15801-21perf07:35:000
411999261,12cyclictest12636-21idleruntime-cro09:55:000
413899259,3cyclictest0-21swapper/310:34:463
413899258,3cyclictest0-21swapper/307:13:333
413899257,2cyclictest265302sleep307:57:063
413899257,2cyclictest265302sleep307:57:063
413899251,10cyclictest0-21swapper/308:49:013
4138992510,2cyclictest0-21swapper/310:38:223
413899250,11cyclictest0-21swapper/307:21:303
413399251,8cyclictest25203-21diskmemload12:37:192
413399251,12cyclictest16743-21ntp_states11:15:232
413399250,13cyclictest0-21swapper/210:23:432
413399250,10cyclictest0-21swapper/210:55:272
412899251,16cyclictest0-21swapper/109:35:161
412899250,23cyclictest0-21swapper/110:15:131
411999250,9cyclictest0-21swapper/009:21:120
411999250,16cyclictest0-21swapper/010:45:190
411999250,11cyclictest0-21swapper/012:25:220
411999250,10cyclictest909-21sssd_nss11:00:170
413899249,3cyclictest0-21swapper/309:13:333
413899249,3cyclictest0-21swapper/307:44:343
413899247,3cyclictest0-21swapper/311:15:053
4138992411,3cyclictest0-21swapper/310:43:503
413899241,12cyclictest0-21swapper/308:35:253
413899241,11cyclictest0-21swapper/311:57:533
413899241,10cyclictest0-21swapper/312:07:183
4133992417,4cyclictest12707-21irqstats12:20:182
413399240,9cyclictest0-21swapper/212:04:362
413399240,11cyclictest0-21swapper/209:10:082
412899248,10cyclictest87550irq/128-enp1s0-10:50:261
4128992411,6cyclictest19055-21kworker/u8:110:20:541
412899241,16cyclictest0-21swapper/112:18:291
412899241,11cyclictest3451-21users09:30:271
411999244,11cyclictest87550irq/128-enp1s0-09:42:060
411999242,12cyclictest24177-40dnf11:34:280
411999242,12cyclictest24177-40dnf11:34:280
411999242,10cyclictest26450-21perf11:40:010
411999240,11cyclictest26048-21/usr/sbin/munin10:25:140
411999240,11cyclictest0-21swapper/010:40:260
413899239,3cyclictest0-21swapper/312:15:343
413899238,3cyclictest0-21swapper/308:21:103
413899232,9cyclictest9349-21sed09:45:193
413899231,11cyclictest0-21swapper/307:15:253
413399231,9cyclictest0-21swapper/211:20:232
412899239,7cyclictest0-21swapper/110:30:211
412899231,17cyclictest0-21swapper/111:29:391
412899231,17cyclictest0-21swapper/111:29:391
412899231,15cyclictest9198-21runrttasks12:11:361
4128992310,10cyclictest20186-21munin-run07:45:001
411999235,10cyclictest10038-21/usr/sbin/munin08:35:240
411999230,15cyclictest0-21swapper/009:05:110
411999230,15cyclictest0-21swapper/009:05:110
411999230,14cyclictest9418-21/usr/sbin/munin07:20:230
411999230,14cyclictest13669-21kworker/u8:210:02:580
411999230,10cyclictest18996-21fschecks_time12:35:140
413899229,3cyclictest0-21swapper/311:45:413
413899227,3cyclictest0-21swapper/311:13:453
413899226,6cyclictest0-21swapper/310:10:433
413899226,3cyclictest0-21swapper/307:39:223
413899225,3cyclictest0-21swapper/309:27:493
4138992210,3cyclictest0-21swapper/309:30:213
413399220,9cyclictest0-21swapper/210:49:112
413399220,9cyclictest0-21swapper/210:26:232
413399220,7cyclictest0-21swapper/207:35:242
412899229,5cyclictest1039-21timerwakeupswit09:25:261
4128992212,4cyclictest14242-21kworker/u8:009:10:091
412899221,18cyclictest25209-21munin-run09:10:011
412899221,18cyclictest25209-21munin-run09:10:011
412899221,15cyclictest87550irq/128-enp1s0-11:18:281
412899221,15cyclictest0-21swapper/111:20:501
412899221,14cyclictest0-21swapper/112:00:371
412899221,12cyclictest28284-21cut09:15:171
412899221,12cyclictest0-21swapper/110:42:101
411999226,4cyclictest80382grep12:10:150
411999221,9cyclictest29596-21dump-pmu-power09:20:010
411999221,9cyclictest14836-21awk10:00:000
4119992216,3cyclictest19482-21cut08:55:170
4119992212,7cyclictest19454-21/usr/sbin/munin10:10:230
411999220,14cyclictest27572-21memory11:40:160
411999220,13cyclictest0-21swapper/012:32:290
411999220,12cyclictest0-21swapper/007:35:260
413899217,3cyclictest0-21swapper/312:36:223
413899217,3cyclictest0-21swapper/311:40:453
413899212,11cyclictest2826-21diskstats10:45:163
413399210,9cyclictest0-21swapper/211:33:122
413399210,9cyclictest0-21swapper/211:33:112
413399210,9cyclictest0-21swapper/210:10:552
413399210,8cyclictest0-21swapper/211:06:032
412899218,11cyclictest23974-21kworker/u8:108:22:571
412899211,5cyclictest0-21swapper/110:29:461
412899211,17cyclictest30776-21df10:35:141
412899211,17cyclictest14656-21kernelversion07:30:181
412899211,11cyclictest790-21in:imjournal09:05:021
412899211,11cyclictest23974-21kworker/u8:109:50:241
412899211,11cyclictest21421-21latency_hist10:15:011
412899211,11cyclictest0-21swapper/111:53:051
412899211,10cyclictest0-21swapper/109:47:381
4128992110,3cyclictest909-21sssd_nss11:55:251
4128992110,3cyclictest17358-21ntp_states12:30:201
411999219,5cyclictest1222-21/usr/sbin/munin08:15:260
411999218,2cyclictest9251-21kworker/0:312:16:210
411999218,2cyclictest88950irq/132-enp2s0-08:03:450
411999218,2cyclictest1350-21kworker/0:209:35:130
411999215,7cyclictest15792-21expr11:15:140
411999214,6cyclictest0-21swapper/011:25:260
411999214,6cyclictest0-21swapper/011:25:260
411999213,11cyclictest0-21swapper/010:31:280
411999211,11cyclictest31402-21/usr/sbin/munin08:10:270
411999210,6cyclictest0-21swapper/010:15:200
411999210,18cyclictest23848-21/usr/sbin/munin10:20:270
411999210,18cyclictest17260-21/usr/sbin/munin10:05:180
411999210,12cyclictest0-21swapper/011:55:220
413899207,1cyclictest0-21swapper/308:15:263
413899205,3cyclictest0-21swapper/308:00:263
413899201,4cyclictest0-21swapper/309:52:473
413899201,14cyclictest25530-21ntp_states11:35:213
413899201,11cyclictest31119-21sshd08:09:283
413899201,11cyclictest20407-21/usr/sbin/munin07:45:273
413899201,11cyclictest0-21swapper/312:12:443
4138992010,1cyclictest0-21swapper/309:40:273
413899200,9cyclictest0-21swapper/309:20:403
413399201,5cyclictest0-21swapper/212:28:392
4133992014,3cyclictest19044-21cstates08:55:132
4128992016,3cyclictest0-21swapper/111:30:521
4128992016,3cyclictest0-21swapper/111:30:521
412899201,18cyclictest0-21swapper/112:21:561
412899201,17cyclictest0-21swapper/112:26:051
412899201,17cyclictest0-21swapper/108:01:231
412899201,17cyclictest0-21swapper/107:45:211
412899201,14cyclictest0-21swapper/112:06:581
412899201,14cyclictest0-21swapper/111:44:341
412899201,14cyclictest0-21swapper/110:08:591
412899201,13cyclictest87550irq/128-enp1s0-11:37:271
412899201,11cyclictest19711-40dnf12:35:231
412899201,11cyclictest16970-21df_inode08:50:151
412899201,11cyclictest13199-21df_inode09:55:141
412899201,11cyclictest0-21swapper/111:05:121
412899201,11cyclictest0-21swapper/110:00:231
412899201,11cyclictest0-21swapper/109:23:541
411999209,3cyclictest1864-21/usr/sbin/munin09:30:210
411999206,4cyclictest5639-21/usr/sbin/munin08:25:150
411999204,5cyclictest87550irq/128-enp1s0-12:25:000
411999202,11cyclictest87550irq/128-enp1s0-08:48:270
411999200,6cyclictest0-21swapper/007:50:190
411999200,13cyclictest0-21swapper/011:05:140
413899195,6cyclictest0-21swapper/310:01:233
413399196,3cyclictest0-21swapper/209:17:282
413399194,6cyclictest0-21swapper/209:45:322
413399191,8cyclictest6661-21awk10:55:012
413399191,7cyclictest30678-21sed08:05:252
413399191,6cyclictest0-21swapper/212:30:132
4133991912,4cyclictest18862-21tail07:40:162
412899191,6cyclictest29930-21ntp_states11:45:241
4128991916,2cyclictest0-21swapper/108:13:321
4128991916,2cyclictest0-21swapper/107:13:251
4128991915,3cyclictest0-21swapper/110:45:191
4128991915,3cyclictest0-21swapper/107:59:511
4128991915,3cyclictest0-21swapper/107:59:511
412899191,17cyclictest0-21swapper/107:29:341
412899191,16cyclictest0-21swapper/111:10:061
412899191,16cyclictest0-21swapper/109:40:041
412899191,16cyclictest0-21swapper/108:27:081
412899191,16cyclictest0-21swapper/108:16:021
4128991911,3cyclictest0-21swapper/111:01:421
412899191,12cyclictest0-21swapper/107:50:181
411999199,2cyclictest5306-21/usr/sbin/munin12:05:200
411999192,9cyclictest13986-21kworker/0:208:51:360
411999192,15cyclictest0-21swapper/007:25:270
4119991912,5cyclictest1633-21munin-run09:30:010
411999191,15cyclictest18207-21/usr/sbin/munin07:40:190
411999190,10cyclictest87550irq/128-enp1s0-10:35:240
411999190,10cyclictest87550irq/128-enp1s0-08:30:270
413899181,6cyclictest0-21swapper/312:26:283
413399187,3cyclictest87350irq/126-enp1s0-08:20:252
413399185,3cyclictest242282cut09:05:202
413399185,3cyclictest242282cut09:05:192
413399181,5cyclictest0-21swapper/210:02:392
4133991812,3cyclictest14851-21fschecks_count08:45:152
413399181,10cyclictest0-21swapper/211:46:042
412899181,9cyclictest15421-21strings08:45:211
4128991815,2cyclictest0-21swapper/107:20:221
4128991814,3cyclictest0-21swapper/107:37:141
412899181,3cyclictest19166-21df08:55:141
412899181,16cyclictest0-21swapper/108:05:031
412899181,15cyclictest0-21swapper/108:35:081
412899180,17cyclictest0-21swapper/110:55:261
411999189,6cyclictest7240-21cpu07:15:140
411999189,6cyclictest3773-21sh08:20:170
411999189,6cyclictest14199-21sh08:45:010
411999182,4cyclictest87350irq/126-enp1s0-07:45:510
411999182,3cyclictest0-21swapper/007:57:590
411999182,3cyclictest0-21swapper/007:57:590
4119991812,3cyclictest27553-21kworker/0:207:10:180
413899175,3cyclictest0-21swapper/307:25:183
413899171,13cyclictest14879-21ntp_states07:30:223
413899171,12cyclictest0-21swapper/308:10:063
413899171,11cyclictest0-21swapper/308:30:223
413899171,10cyclictest0-21swapper/309:35:593
413399175,3cyclictest87350irq/126-enp1s0-12:05:162
413399175,3cyclictest32345-21missed_timers08:10:202
413399171,9cyclictest0-21swapper/211:56:022
413399171,9cyclictest0-21swapper/211:51:072
413399171,9cyclictest0-21swapper/210:07:012
413399171,8cyclictest0-21swapper/208:53:232
413399171,7cyclictest30888-21ntp_states09:20:222
413399171,6cyclictest11388-21latency_hist07:25:022
413399171,6cyclictest10259-21perl12:15:182
413399171,5cyclictest0-21swapper/211:26:552
413399171,5cyclictest0-21swapper/211:26:552
413399171,3cyclictest87350irq/126-enp1s0-09:35:382
413399171,3cyclictest0-21swapper/208:19:432
4133991712,3cyclictest32725-21idleruntime09:25:182
4133991712,3cyclictest24610-21cat07:55:012
4133991712,3cyclictest13972-21tr07:30:132
4133991712,3cyclictest12855-21/usr/sbin/munin09:55:272
4133991712,3cyclictest12272-21cut07:25:182
413399171,15cyclictest0-21swapper/207:12:432
413399171,14cyclictest0-21swapper/207:55:192
413399171,14cyclictest0-21swapper/207:55:192
4133991711,3cyclictest20682-21cat07:45:142
412899171,10cyclictest0-21swapper/108:30:461
226932170,2sleep20-21swapper/209:01:102
413399162,2cyclictest87350irq/126-enp1s0-12:10:262
413399161,6cyclictest6591-21entropy07:15:022
413399161,6cyclictest27047-21cpu08:00:112
413399161,6cyclictest0-21swapper/209:40:072
413399161,5cyclictest0-21swapper/209:50:142
4133991612,3cyclictest0-21swapper/208:35:062
4133991611,3cyclictest7840-21/usr/sbin/munin08:30:142
4133991610,3cyclictest24562-21grep11:35:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional