You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-20 - 03:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Sun Sep 20, 2026 00:44:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
294822229162,54sleep20-21swapper/219:05:092
312072212183,19sleep10-21swapper/119:07:011
312702202164,25sleep00-21swapper/019:07:490
313252201165,24sleep30-21swapper/319:08:323
2739021440,3sleep13152799cyclictest21:25:141
325052800,1sleep30-21swapper/300:05:013
123122560,3sleep33153999cyclictest19:35:253
157232540,1sleep20-21swapper/222:10:262
132122520,4sleep33153999cyclictest00:33:583
31522994010,14cyclictest11113-21munin-run00:30:010
3152299377,14cyclictest28406-21munin-run20:15:010
49562350,1sleep00-21swapper/020:35:010
3152799351,26cyclictest0-21swapper/120:35:171
3152799351,11cyclictest0-21swapper/120:49:371
31522993510,12cyclictest20852-21munin-run22:25:010
3152299346,13cyclictest23057-21awk22:30:010
3152299334,13cyclictest16456-21munin-run22:15:010
3152299331,22cyclictest13950-21kworker/0:122:42:010
3152299314,12cyclictest12073-21awk22:05:010
3152299311,10cyclictest25536-21kworker/0:000:23:410
3152299301,19cyclictest13950-21kworker/0:122:36:090
3152799291,12cyclictest0-21swapper/122:31:491
3152799290,5cyclictest0-21swapper/100:12:531
3152799290,13cyclictest0-21swapper/121:11:591
3152799290,11cyclictest0-21swapper/121:18:411
93832280,4sleep09386-21fschecks_time19:30:160
93832280,4sleep09386-21fschecks_time19:30:160
3152799281,12cyclictest0-21swapper/121:30:331
3152799281,11cyclictest0-21swapper/120:19:451
3152799281,11cyclictest0-21swapper/119:40:491
3152299281,18cyclictest13950-21kworker/0:123:19:490
166822280,3chrt16681-21gltestperf21:00:170
3153999271,16cyclictest0-21swapper/321:53:253
3152799272,17cyclictest0-21swapper/100:22:251
3152799271,11cyclictest0-21swapper/123:42:571
3152799271,11cyclictest0-21swapper/123:06:211
3152799271,11cyclictest0-21swapper/123:01:011
3152799270,13cyclictest0-21swapper/121:37:451
3152799270,11cyclictest0-21swapper/122:09:331
3152299272,15cyclictest31578-21kworker/0:221:22:450
3152299271,9cyclictest13950-21kworker/0:123:30:570
3152299271,11cyclictest13950-21kworker/0:122:32:330
3153999260,12cyclictest0-21swapper/319:40:473
3152799261,9cyclictest0-21swapper/119:14:571
3152799261,8cyclictest0-21swapper/119:39:131
3152799261,8cyclictest0-21swapper/119:33:291
3152799261,8cyclictest0-21swapper/119:33:291
3152799261,7cyclictest0-21swapper/120:58:491
3152799261,11cyclictest0-21swapper/122:26:251
3152799261,11cyclictest0-21swapper/121:07:211
3152799260,10cyclictest0-21swapper/121:02:091
3152299260,20cyclictest13950-21kworker/0:123:21:170
316122251,3sleep10-21swapper/122:47:181
3153299251,8cyclictest2324-21munin-run00:10:012
3152799251,9cyclictest0-21swapper/123:21:331
3152799251,9cyclictest0-21swapper/122:17:021
3152799251,9cyclictest0-21swapper/119:52:051
31527992519,3cyclictest9919-21/usr/sbin/munin00:25:201
3152799251,8cyclictest0-21swapper/121:47:491
3152799251,7cyclictest0-21swapper/100:09:451
3152799251,22cyclictest0-21swapper/122:13:451
3152799251,17cyclictest0-21swapper/123:35:011
3152799251,11cyclictest0-21swapper/121:55:531
3152799251,11cyclictest0-21swapper/120:32:351
3152799251,11cyclictest0-21swapper/119:57:331
3152799251,11cyclictest0-21swapper/119:19:211
3152799251,10cyclictest0-21swapper/122:52:091
3152799251,10cyclictest0-21swapper/120:52:451
3152799251,10cyclictest0-21swapper/120:44:371
3152799251,10cyclictest0-21swapper/100:16:571
3152299251,9cyclictest13950-21kworker/0:123:03:090
3152299251,8cyclictest5430-21kworker/0:221:58:410
31539992413,6cyclictest12923-21sensors20:50:233
3153999240,2cyclictest0-21swapper/323:21:463
3153299249,3cyclictest0-21swapper/222:36:292
3153299243,5cyclictest0-21swapper/222:27:022
3153299241,5cyclictest0-21swapper/221:25:172
3152799242,19cyclictest27827-40dnf23:52:201
3152799241,9cyclictest0-21swapper/122:20:141
3152799241,9cyclictest0-21swapper/121:54:051
3152799241,14cyclictest0-21swapper/123:55:071
3152799241,11cyclictest0-21swapper/119:28:471
3152799241,10cyclictest11958-21chrt22:03:511
3152799241,10cyclictest0-21swapper/120:13:311
3152799241,10cyclictest0-21swapper/120:00:591
3152799240,12cyclictest0-21swapper/122:36:491
3152799240,11cyclictest0-21swapper/123:48:171
3152799240,11cyclictest0-21swapper/119:24:511
3152799240,10cyclictest0-21swapper/122:40:291
3152299242,9cyclictest8222-21kworker/0:019:13:150
3152299241,9cyclictest13950-21kworker/0:123:25:330
3152299240,9cyclictest13950-21kworker/0:122:48:210
3153999236,3cyclictest0-21swapper/322:30:393
3153999235,3cyclictest0-21swapper/323:08:303
3153999232,13cyclictest12628-21df_inode22:05:143
3152799237,8cyclictest0-21swapper/123:38:221
3152799231,9cyclictest0-21swapper/100:04:371
3152799231,8cyclictest0-21swapper/120:23:331
3152799231,4cyclictest0-21swapper/100:37:361
3152799231,14cyclictest0-21swapper/121:43:001
3152799231,10cyclictest0-21swapper/120:28:471
3152799231,10cyclictest0-21swapper/119:47:271
3152299238,3cyclictest31578-21kworker/0:220:49:070
3152299234,16cyclictest4241-21idleruntime-cro19:20:010
3152299232,9cyclictest31578-21kworker/0:221:26:150
3152299232,9cyclictest31578-21kworker/0:221:12:430
3152299232,8cyclictest8222-21kworker/0:019:36:070
3152299232,8cyclictest25536-21kworker/0:023:57:090
3152299231,8cyclictest25536-21kworker/0:000:00:250
3152299231,7cyclictest13950-21kworker/0:123:05:130
3152299231,5cyclictest0-21swapper/021:35:390
3152299230,11cyclictest7670-21idleruntime-cro21:55:010
198202231,3sleep019833-21diskstats23:35:130
3153999222,7cyclictest28312-21smart_sda21:25:253
31539992216,4cyclictest26747-21cat22:35:243
3152799221,9cyclictest0-21swapper/123:12:111
3152799221,7cyclictest26198-21kworker/u8:221:23:281
3152799221,18cyclictest12011-21hddtemp_smartct00:30:181
3152799221,10cyclictest0-21swapper/120:05:551
3152799220,8cyclictest0-21swapper/123:28:331
3152299222,11cyclictest30828-21/usr/sbin/munin20:20:290
3152299221,8cyclictest21820-21kworker/0:120:08:430
3152299221,11cyclictest25536-21kworker/0:023:50:340
3152299221,10cyclictest0-21swapper/021:18:190
61272210,2sleep00-21swapper/020:35:220
3153999219,3cyclictest23720-21ntp_states21:15:223
3153999213,9cyclictest0-21swapper/322:47:553
3153999212,5cyclictest1364-40dnf22:51:233
3153999211,17cyclictest8001-21ntpq00:20:213
3153999211,14cyclictest0-21swapper/300:05:493
31539992111,3cyclictest11017-21sed20:45:263
3153999211,12cyclictest0-21swapper/321:30:143
3153999211,11cyclictest3548-21cpuspeed_turbos21:45:123
3153299215,9cyclictest12722-40dnf20:50:232
3153299211,6cyclictest0-21swapper/223:06:512
3153299211,6cyclictest0-21swapper/222:46:412
3153299211,5cyclictest0-21swapper/223:52:222
3153299211,5cyclictest0-21swapper/200:18:042
3152799217,3cyclictest0-21swapper/123:15:471
3152299218,3cyclictest8222-21kworker/0:019:20:410
3152299218,2cyclictest31578-21kworker/0:221:06:350
3152299218,2cyclictest13950-21kworker/0:123:44:510
3152299217,3cyclictest31578-21kworker/0:220:51:590
3152299213,6cyclictest8564-1kworker/u9:122:17:400
3152299212,8cyclictest13950-21kworker/0:122:50:030
3152299211,8cyclictest13950-21kworker/0:122:07:330
3152299211,4cyclictest0-21swapper/022:57:100
3152299211,11cyclictest25536-21kworker/0:000:13:050
3152299210,16cyclictest13950-21kworker/0:123:45:290
3153999208,4cyclictest0-21swapper/321:20:203
3153999207,3cyclictest20356-21latency23:35:173
3153999206,7cyclictest13868-21df00:35:153
3153999205,9cyclictest0-21swapper/323:42:143
3153999205,3cyclictest0-21swapper/320:33:253
3153999204,3cyclictest0-21swapper/323:13:493
3153999201,12cyclictest0-21swapper/321:38:403
3153999201,10cyclictest19331-21ntp_states21:05:223
3153999201,10cyclictest0-21swapper/322:43:053
3153999201,10cyclictest0-21swapper/320:01:493
3153999200,11cyclictest0-21swapper/323:56:373
3153299201,8cyclictest15442-21/usr/sbin/munin19:45:242
3153299201,7cyclictest0-21swapper/200:11:042
31532992017,2cyclictest0-21swapper/221:19:152
3153299201,6cyclictest0-21swapper/223:35:162
3153299201,5cyclictest0-21swapper/223:59:152
3153299201,5cyclictest0-21swapper/221:21:002
3153299201,4cyclictest0-21swapper/223:30:592
3153299201,4cyclictest0-21swapper/223:23:462
3153299201,4cyclictest0-21swapper/221:57:562
3153299201,3cyclictest0-21swapper/222:20:372
3152799209,4cyclictest0-21swapper/122:56:131
3152299208,2cyclictest31050-21kworker/0:121:43:590
3152299206,3cyclictest808-21snmpd19:25:470
3152299206,3cyclictest31578-21kworker/0:220:25:070
3152299206,2cyclictest17324-21kworker/0:219:58:270
3152299206,2cyclictest17324-21kworker/0:219:50:550
3152299202,9cyclictest25536-21kworker/0:000:08:110
3152299202,8cyclictest31050-21kworker/0:121:45:480
3152299201,14cyclictest8222-21kworker/0:019:45:450
3152299201,11cyclictest9091-21sshd20:43:200
3153999197,3cyclictest0-21swapper/322:25:183
3153999196,3cyclictest8564-1kworker/u9:123:01:043
3153999196,3cyclictest0-21swapper/323:45:033
3153999196,3cyclictest0-21swapper/322:17:413
3153999195,7cyclictest755-21rngd22:24:213
3153999194,5cyclictest0-21swapper/323:30:183
3153999194,12cyclictest20523-21cpuspeed_turbos21:10:143
3153999192,15cyclictest401ktimersoftd/320:05:283
3153999191,9cyclictest27719-21seq23:50:573
3153999191,8cyclictest0-21swapper/300:10:063
3153999191,6cyclictest0-21swapper/300:25:203
31539991914,3cyclictest3982chrt20:25:013
3153999191,15cyclictest5752-21switchtime19:20:253
31539991911,3cyclictest0-21swapper/321:55:093
3153999191,10cyclictest19837-21/usr/sbin/munin19:55:143
3153999191,10cyclictest10081-21/usr/sbin/munin22:00:183
3153999190,10cyclictest3386-21unixbench-2d22:55:263
3153299197,9cyclictest1-21systemd23:00:022
3153299195,3cyclictest12050irq/123-ahci[0019:55:022
3153299191,6cyclictest0-21swapper/223:15:222
3153299191,6cyclictest0-21swapper/222:50:152
3153299191,6cyclictest0-21swapper/200:35:292
3153299191,5cyclictest1595-21sort19:15:022
3153299191,5cyclictest0-21swapper/222:04:362
3153299191,4cyclictest0-21swapper/200:25:132
31532991913,3cyclictest5119-21gltestperf19:20:162
3152299196,3cyclictest87550irq/128-enp1s0-00:15:210
3152299196,2cyclictest31578-21kworker/0:220:58:350
3152299195,5cyclictest14344-21sshd00:35:200
3152299195,3cyclictest13950-21kworker/0:123:12:570
3152299195,2cyclictest25536-21kworker/0:000:30:210
3152299193,3cyclictest15219-21munin-run19:45:010
3152299191,9cyclictest31578-21kworker/0:221:30:350
162702190,7sleep2311ktimersoftd/223:25:232
53892181,2sleep20-21swapper/221:49:202
3153999188,3cyclictest18702-21ntp_states19:50:203
3153999186,8cyclictest0-21swapper/323:29:273
3153999181,9cyclictest0-21swapper/321:40:503
3153999181,6cyclictest28486-21date20:15:023
31539991813,3cyclictest5560-21irqstats00:15:193
3153999181,14cyclictest6625-21sed20:35:263
3153999181,12cyclictest0-21swapper/323:16:053
31539991810,6cyclictest15544-21ntpq22:10:243
3153999180,12cyclictest0-21swapper/319:28:443
3153299184,3cyclictest7357-21sshd21:51:202
3153299182,3cyclictest24915-21grep23:45:212
3153299182,3cyclictest0-21swapper/221:37:222
3153299182,3cyclictest0-21swapper/221:06:362
3153299182,13cyclictest11709-21cut19:35:162
3153299181,7cyclictest0-21swapper/219:43:482
3153299181,6cyclictest0-21swapper/221:42:412
3153299181,5cyclictest30828-21/usr/sbin/munin20:20:192
3153299181,5cyclictest30486-21/usr/sbin/munin00:00:252
3153299181,5cyclictest22363-21sshd21:14:202
3153299181,3cyclictest0-21swapper/223:42:162
31532991811,4cyclictest7921-21cut20:40:152
31532991811,4cyclictest29247-21tail20:15:162
3153299181,13cyclictest0-21swapper/220:56:112
3153999172,6cyclictest0-21swapper/320:40:133
3153999172,12cyclictest28632-21/usr/sbin/munin20:15:213
3153999171,6cyclictest0-21swapper/319:45:153
31539991712,3cyclictest2862-21tail19:15:163
31539991711,3cyclictest1852-21processes20:25:203
3153999170,4cyclictest0-21swapper/319:15:013
3153999170,10cyclictest0-21swapper/320:55:233
3153299177,8cyclictest579-21systemd-journal20:35:012
3153299171,7cyclictest23271-21/usr/sbin/munin22:30:222
3153299171,6cyclictest4007-21sshd19:17:202
3153299171,6cyclictest0-21swapper/222:06:282
3153299171,6cyclictest0-21swapper/221:33:062
3153299171,6cyclictest0-21swapper/220:00:132
3153299171,4cyclictest7192-21cat20:40:012
3153299171,3cyclictest0-21swapper/223:10:052
31532991713,3cyclictest0-21swapper/221:03:302
3153299171,2cyclictest87650irq/129-enp1s0-19:30:122
3153299171,2cyclictest87650irq/129-enp1s0-19:30:122
31532991712,3cyclictest25579-21threads20:05:272
31532991711,4cyclictest28895-21/usr/sbin/munin22:40:232
31532991711,3cyclictest7028-21cpuspeed_turbos00:20:122
31532991711,3cyclictest27021-21grep20:10:152
31532991711,3cyclictest16844-21cut22:15:122
3153299170,15cyclictest0-21swapper/200:30:242
104452170,2sleep30-21swapper/319:30:283
104452170,2sleep30-21swapper/319:30:283
31539991610,4cyclictest17586-21grep21:00:273
3153299161,6cyclictest2342-21sshd20:26:212
31532991612,3cyclictest0-21swapper/220:46:292
31532991611,3cyclictest5336-21sendmail_mailqu23:00:242
31532991611,3cyclictest20178-21sh19:55:152
3152299161,2cyclictest21820-21kworker/0:120:15:130
3152299161,14cyclictest0-21swapper/020:00:070
3153299151,4cyclictest0-21swapper/219:25:202
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional