You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 14:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Tue Aug 25, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13502238162,24sleep30-21swapper/307:06:303
13692237163,23sleep20-21swapper/207:06:452
12802203165,25sleep10-21swapper/107:05:371
14542202167,24sleep00-21swapper/007:07:530
1020621640,3sleep2173599cyclictest11:05:242
491521410,3sleep1172999cyclictest09:40:241
1578921330,4sleep3174199cyclictest11:20:143
172199629,13cyclictest8014-21munin-run09:50:020
266062470,1sleep20-21swapper/208:05:002
172199445,12cyclictest9598-21munin-run08:40:010
172199368,13cyclictest28776-21munin-run08:10:010
172199367,14cyclictest18395-21munin-run09:00:010
172199358,12cyclictest12050irq/123-ahci[0007:30:020
172199334,14cyclictest720-21munin-run08:20:000
173599321,23cyclictest0-21swapper/211:38:332
173599321,22cyclictest22792-21diskmemload12:32:142
173599321,18cyclictest0-21swapper/208:32:222
172999322,25cyclictest17111-21ls10:10:131
1721993210,10cyclictest1410-21perf09:35:010
1721993210,10cyclictest1410-21perf09:35:010
173599311,17cyclictest0-21swapper/211:15:052
173599311,12cyclictest0-21swapper/212:00:082
172199316,12cyclictest24991-21munin-run09:15:010
173599303,11cyclictest8011-21grep07:20:192
173599301,2cyclictest0-21swapper/210:55:532
173599301,17cyclictest0-21swapper/211:31:172
173599301,13cyclictest0-21swapper/208:18:202
173599301,11cyclictest0-21swapper/212:14:382
173599301,11cyclictest0-21swapper/209:05:472
173599301,10cyclictest0-21swapper/207:30:272
173599300,20cyclictest0-21swapper/212:20:062
172999301,17cyclictest26782-21df11:45:131
172199307,11cyclictest3634-21perf09:40:010
173599292,4cyclictest648-21hddtemp_smartct10:45:162
173599292,4cyclictest648-21hddtemp_smartct10:45:162
173599291,22cyclictest0-21swapper/210:00:382
173599291,12cyclictest0-21swapper/211:47:212
173599291,11cyclictest0-21swapper/211:01:292
173599291,10cyclictest18585-21cat11:25:192
172199292,10cyclictest2366-21kworker/0:112:02:490
173599287,11cyclictest0-21swapper/208:55:302
173599281,9cyclictest0-21swapper/212:18:582
173599281,7cyclictest0-21swapper/208:53:142
173599281,18cyclictest0-21swapper/210:13:022
173599281,17cyclictest0-21swapper/208:45:502
173599281,15cyclictest0-21swapper/207:51:172
173599281,13cyclictest0-21swapper/210:32:542
173599281,13cyclictest0-21swapper/207:37:212
173599281,12cyclictest0-21swapper/209:37:252
173599280,9cyclictest0-21swapper/212:28:442
173599280,9cyclictest0-21swapper/209:22:292
173599280,18cyclictest0-21swapper/209:56:092
173599280,17cyclictest0-21swapper/210:07:142
173599280,13cyclictest0-21swapper/211:23:492
173599280,12cyclictest0-21swapper/210:36:042
173599280,12cyclictest0-21swapper/210:22:162
173599280,12cyclictest0-21swapper/208:25:182
173599280,11cyclictest0-21swapper/209:50:412
1729992813,4cyclictest26503-21head10:30:151
172199283,12cyclictest31579-21munin-run09:30:010
172199282,9cyclictest17038-40dnf10:10:120
173599271,17cyclictest0-21swapper/207:57:532
173599271,11cyclictest4803-21taskset12:09:262
173599271,11cyclictest0-21swapper/211:58:042
173599271,11cyclictest0-21swapper/209:27:232
173599271,11cyclictest0-21swapper/208:43:062
173599271,11cyclictest0-21swapper/208:23:042
173599271,11cyclictest0-21swapper/208:23:042
173599271,11cyclictest0-21swapper/207:26:412
173599271,10cyclictest0-21swapper/207:16:102
173599270,16cyclictest0-21swapper/209:48:292
173599270,11cyclictest0-21swapper/211:53:122
173599270,11cyclictest0-21swapper/210:25:462
172199272,11cyclictest24036-21kworker/0:007:58:380
172199271,10cyclictest15203-21kworker/0:111:22:100
173599261,18cyclictest0-21swapper/211:12:052
173599261,18cyclictest0-21swapper/210:51:132
173599261,10cyclictest0-21swapper/212:39:302
173599261,10cyclictest0-21swapper/210:43:302
173599261,10cyclictest0-21swapper/208:38:102
173599261,10cyclictest0-21swapper/208:06:352
173599261,10cyclictest0-21swapper/207:11:382
173599260,20cyclictest0-21swapper/210:16:182
173599260,13cyclictest0-21swapper/209:44:372
173599260,11cyclictest0-21swapper/207:40:312
172199262,10cyclictest18105-21kworker/0:109:18:300
172199262,10cyclictest12980-21kworker/0:007:12:170
1721992612,2cyclictest17763-21kworker/0:212:37:490
172199261,12cyclictest7-21ksoftirqd/011:50:010
172199261,10cyclictest19782-21kworker/0:307:49:100
172199260,12cyclictest27815-21awk10:35:000
1741992512,3cyclictest28406-21idleruntime-cro11:50:003
173599252,5cyclictest19968-21idleruntime-cro07:50:012
173599251,9cyclictest0-21swapper/209:16:472
172999250,19cyclictest0-21swapper/108:02:361
172199252,10cyclictest2974-21kworker/0:008:26:130
172199251,12cyclictest9272-21perf12:20:010
172199251,12cyclictest16819-21munin-run10:10:010
172199251,11cyclictest27603-21kworker/0:210:36:050
172199251,10cyclictest6869-21kworker/0:012:14:010
172199251,10cyclictest14493-21kworker/0:010:05:010
1721992510,3cyclictest27603-21kworker/0:211:00:100
172199250,9cyclictest12965-21kworker/0:011:15:540
172199250,12cyclictest11809-21awk08:45:010
174199241,7cyclictest0-21swapper/309:40:283
173599245,3cyclictest0-21swapper/209:30:532
173599245,3cyclictest0-21swapper/209:30:532
173599240,9cyclictest0-21swapper/208:10:422
173599240,12cyclictest0-21swapper/209:04:052
172999249,13cyclictest26881-21kworker/u8:009:18:401
172999241,12cyclictest22506-21ntp_states10:20:201
1729992410,3cyclictest4070-21ntp_states07:15:021
172999240,9cyclictest0-21swapper/110:03:161
172999240,17cyclictest0-21swapper/110:35:111
172199242,11cyclictest25622-21idleruntime-cro10:30:000
172199242,11cyclictest0-21swapper/007:35:020
172199242,10cyclictest21496-21egrep10:20:100
1721992411,3cyclictest29226-21kworker/0:309:40:180
172199241,12cyclictest13052-21perf11:15:020
172199241,11cyclictest6697-21kworker/0:107:24:010
1721992410,3cyclictest32482-21kworker/0:011:57:090
1721992410,3cyclictest15845-21kworker/0:308:53:330
35292230,10sleep011-21rcuc/010:50:260
173599230,10cyclictest0-21swapper/209:14:372
172999230,16cyclictest0-21swapper/112:00:021
172999230,16cyclictest0-21swapper/108:10:211
172199231,11cyclictest2054-21basename10:50:000
172199231,11cyclictest2054-21basename10:50:000
172199231,10cyclictest19782-21kworker/0:307:52:220
1721992310,2cyclictest12965-21kworker/0:011:31:500
1721992310,2cyclictest11461-21kworker/0:008:46:210
172199230,9cyclictest2974-21kworker/0:008:34:450
174199227,2cyclictest0-21swapper/307:34:453
174199225,4cyclictest10830-21crond11:10:023
174199224,9cyclictest19855-21/usr/sbin/munin11:30:103
1741992215,4cyclictest335-21latency_hist12:00:003
1741992210,3cyclictest19582-21df_inode10:15:133
172999222,15cyclictest0-21swapper/107:38:361
1729992218,3cyclictest0-21swapper/108:32:191
172999221,3cyclictest31540-21diskstats08:15:141
172999221,15cyclictest0-21swapper/111:35:481
172999221,15cyclictest0-21swapper/107:54:441
172999221,15cyclictest0-21swapper/107:40:501
172199229,2cyclictest24036-21kworker/0:008:04:540
172199229,10cyclictest3267-21fschecks_count12:05:160
172199228,2cyclictest18105-21kworker/0:109:04:250
172199227,4cyclictest2366-21kworker/0:112:22:530
172199227,4cyclictest21608-21kworker/0:111:37:020
172199227,2cyclictest31964-21kworker/0:310:44:490
172199226,3cyclictest87650irq/129-enp1s0-09:55:160
1721992211,3cyclictest9076-21kworker/0:312:25:050
172199221,10cyclictest18105-21kworker/0:109:08:100
174199218,3cyclictest7034-21head09:45:183
174199215,8cyclictest87350irq/126-enp1s0-09:30:003
1741992111,2cyclictest14621-21awk10:05:013
172999216,3cyclictest30322-21basename09:25:171
172999216,12cyclictest25600-21latency_hist10:30:021
172999212,14cyclictest0-21swapper/107:48:331
172999212,14cyclictest0-21swapper/107:27:181
172999212,10cyclictest0-21swapper/112:17:101
172999211,8cyclictest0-21swapper/109:39:551
1729992113,3cyclictest0-21swapper/111:51:331
172999211,17cyclictest16563-21/usr/sbin/munin11:20:221
172999211,15cyclictest0-21swapper/107:32:221
172999211,11cyclictest7112-21ntp_states09:45:221
172999211,11cyclictest3964-21ntp_states12:05:241
172999211,11cyclictest24583-21df11:40:141
172999211,11cyclictest10207-21latency_hist09:55:021
172199219,3cyclictest645-21kworker/0:108:22:450
172199219,3cyclictest645-21kworker/0:108:22:450
172199218,2cyclictest9076-21kworker/0:312:31:250
172199216,4cyclictest5693-21kworker/0:109:50:380
172199212,9cyclictest12965-21kworker/0:011:27:460
172199211,6cyclictest0-21swapper/011:08:510
172199211,12cyclictest26326-21kworker/0:208:12:370
172199211,12cyclictest13045-21kworker/0:207:36:260
172199211,10cyclictest23854-21kworker/0:211:54:530
174199204,9cyclictest39-21rcuc/312:05:213
174199203,7cyclictest5843-21iwlist12:10:163
174199202,5cyclictest6445-21perf11:00:023
174199201,7cyclictest0-21swapper/310:52:103
174199201,11cyclictest24592-21/usr/sbin/munin08:00:153
174199201,10cyclictest18138-21open_inodes10:10:223
174199201,10cyclictest0-21swapper/309:15:123
173599200,9cyclictest0-21swapper/211:44:252
172999203,7cyclictest0-21swapper/112:00:191
172999202,16cyclictest0-21swapper/108:35:091
1729992016,3cyclictest0-21swapper/111:26:181
1729992013,3cyclictest0-21swapper/108:08:291
1729992012,3cyclictest0-21swapper/111:09:301
172999201,18cyclictest0-21swapper/112:10:541
172999201,18cyclictest0-21swapper/108:56:111
172999201,17cyclictest0-21swapper/112:39:121
172999201,17cyclictest0-21swapper/110:43:221
172999201,17cyclictest0-21swapper/110:16:061
172999201,14cyclictest0-21swapper/108:20:501
172999201,14cyclictest0-21swapper/108:20:501
172999201,10cyclictest14953-21ntp_states12:30:241
172999201,10cyclictest14332-21ntp_states11:15:211
172999201,10cyclictest12167-40dnf11:10:221
172999200,15cyclictest0-21swapper/110:58:231
172199208,3cyclictest4826-21/usr/sbin/munin07:15:230
172199206,5cyclictest0-21swapper/010:59:160
172199201,6cyclictest0-21swapper/009:25:000
1721992014,3cyclictest24882-21tail11:40:170
72812190,2sleep30-21swapper/312:15:093
250102190,3sleep325013-21latency_hist09:15:013
174199198,3cyclictest808-21snmpd12:33:063
174199198,2cyclictest12944-21smart_sda12:25:233
174199198,2cyclictest0-21swapper/309:55:123
174199193,9cyclictest0-21swapper/312:24:163
174199193,9cyclictest0-21swapper/310:09:283
174199193,6cyclictest755-21rngd12:00:413
174199193,4cyclictest7407-21perf08:35:023
174199192,4cyclictest9103-21cat09:50:183
174199191,6cyclictest0-21swapper/311:10:453
174199191,5cyclictest4072-21sed07:15:013
174199191,10cyclictest18152-21sshd08:57:113
174199191,10cyclictest17699-21sshd07:44:103
174199191,10cyclictest0-21swapper/307:20:233
172999198,3cyclictest11088-21smart_sda08:40:231
172999195,12cyclictest3944-21kworker/u8:209:10:311
172999192,3cyclictest0-21swapper/109:06:481
1729991915,3cyclictest0-21swapper/109:57:291
1729991915,3cyclictest0-21swapper/108:52:361
172999191,17cyclictest0-21swapper/112:28:021
172999191,17cyclictest0-21swapper/110:54:131
172999191,16cyclictest0-21swapper/111:30:581
172999191,14cyclictest0-21swapper/111:03:161
172999191,10cyclictest19692-21ntpq09:00:221
174199189,2cyclictest0-21swapper/311:41:323
174199188,3cyclictest20204-21/usr/sbin/munin07:50:263
174199188,2cyclictest22398-21/usr/sbin/munin07:55:163
174199188,2cyclictest1031-21cut10:45:193
174199188,2cyclictest1031-21cut10:45:193
174199183,12cyclictest39-21rcuc/309:05:213
174199181,9cyclictest808-21snmpd10:37:443
174199181,8cyclictest15760-21latency_hist07:40:013
174199181,6cyclictest0-21swapper/311:29:103
1741991811,4cyclictest32503-21idleruntime09:30:173
1741991811,4cyclictest32503-21idleruntime09:30:173
174199181,10cyclictest0-21swapper/310:40:063
174199181,10cyclictest0-21swapper/310:27:583
172999187,3cyclictest0-21swapper/108:45:141
172999181,8cyclictest0-21swapper/110:48:371
172999181,8cyclictest0-21swapper/110:48:371
1729991814,3cyclictest0-21swapper/110:06:081
1729991812,2cyclictest0-21swapper/107:22:021
172999181,16cyclictest0-21swapper/109:24:301
172999181,11cyclictest0-21swapper/107:55:171
172999181,10cyclictest0-21swapper/112:20:161
172999180,1cyclictest0-21swapper/107:15:121
172999180,10cyclictest0-21swapper/109:30:241
172999180,10cyclictest0-21swapper/109:30:241
172199181,10cyclictest13045-21kworker/0:207:40:540
172199181,10cyclictest0-21swapper/010:19:480
237932171,2sleep30-21swapper/311:37:163
174199178,2cyclictest26801-21/usr/sbin/munin08:05:213
174199177,3cyclictest87350irq/126-enp1s0-07:48:313
174199175,3cyclictest13307-21cpu11:15:133
174199175,2cyclictest87350irq/126-enp1s0-10:23:383
174199175,2cyclictest87350irq/126-enp1s0-08:10:203
174199173,11cyclictest39-21rcuc/311:50:203
174199172,9cyclictest839-21lldpd08:36:493
174199171,9cyclictest0-21swapper/309:35:103
174199171,4cyclictest5091-21cstates07:15:133
1741991712,3cyclictest27415-21/usr/sbin/munin09:20:213
1741991712,3cyclictest26686-21wc10:30:173
1741991712,3cyclictest16382-21cat08:55:023
174199171,15cyclictest0-21swapper/308:27:303
174199171,13cyclictest2550-21sed08:20:243
174199171,13cyclictest2550-21sed08:20:243
172999170,16cyclictest0-21swapper/108:26:131
174199166,3cyclictest87350irq/126-enp1s0-08:15:113
174199162,5cyclictest6731-21ls11:00:143
174199161,6cyclictest0-21swapper/312:35:043
174199161,5cyclictest579-21systemd-journal09:05:023
174199161,4cyclictest808-21snmpd07:27:103
1741991611,3cyclictest12374-21dnf08:45:153
1741991611,3cyclictest10171-21df_inode08:40:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional