You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-07 - 14:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Mon Sep 07, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
324692236163,24sleep20-21swapper/219:08:532
324492234162,22sleep30-21swapper/319:08:393
324572203167,24sleep10-21swapper/119:08:441
323022200162,25sleep00-21swapper/019:06:420
3039721820,5sleep33265499cyclictest20:15:153
6532590,2sleep20-21swapper/222:50:012
208072590,3sleep011-21rcuc/022:20:180
17322520,4sleep03263499cyclictest19:10:240
193552480,3sleep10-21swapper/122:15:251
193552480,3sleep10-21swapper/122:15:251
3263499377,14cyclictest22916-21munin-run20:00:020
3263499377,14cyclictest22916-21munin-run20:00:020
32634993710,13cyclictest13788-21munin-run23:20:000
3263499369,13cyclictest7209-21awk23:05:010
3263499356,13cyclictest20721-21munin-run19:55:000
3265499331,20cyclictest0-21swapper/320:49:133
3263499335,13cyclictest24156-21munin-run22:30:010
3265499301,22cyclictest0-21swapper/319:31:023
3265499301,19cyclictest0-21swapper/319:57:303
3265499301,19cyclictest0-21swapper/319:57:303
3265499301,19cyclictest0-21swapper/319:11:453
3265499301,10cyclictest0-21swapper/323:55:103
3265499300,24cyclictest0-21swapper/322:01:023
3265499300,18cyclictest0-21swapper/322:26:343
3265499300,18cyclictest0-21swapper/320:12:503
3265499299,2cyclictest0-21swapper/323:36:593
3265499291,11cyclictest13010-21expr20:50:133
3265499290,22cyclictest0-21swapper/322:19:303
3265499290,22cyclictest0-21swapper/322:19:303
3265499290,17cyclictest0-21swapper/321:28:213
3265499290,10cyclictest0-21swapper/321:41:333
3263499291,13cyclictest18524-21munin-run19:50:000
32634992910,16cyclictest15986-21munin-run23:25:000
3265499281,9cyclictest0-21swapper/323:51:033
3265499281,9cyclictest0-21swapper/319:43:263
3265499281,10cyclictest0-21swapper/300:20:023
3265499280,13cyclictest0-21swapper/319:27:583
3265499280,12cyclictest0-21swapper/321:37:213
3263499289,16cyclictest27322-21munin-run20:10:020
3263499282,11cyclictest16337-21basename19:45:000
3265499271,9cyclictest0-21swapper/321:19:573
3265499271,9cyclictest0-21swapper/320:40:293
3265499271,9cyclictest0-21swapper/320:26:253
3265499271,9cyclictest0-21swapper/320:09:413
3265499271,9cyclictest0-21swapper/319:36:143
3265499271,11cyclictest0-21swapper/323:14:173
3265499271,11cyclictest0-21swapper/322:38:183
3265499271,10cyclictest0-21swapper/323:33:173
3265499271,10cyclictest0-21swapper/300:37:263
3265499270,13cyclictest0-21swapper/322:57:053
3265499270,12cyclictest0-21swapper/322:31:303
3265499270,12cyclictest0-21swapper/321:56:013
3265499270,11cyclictest0-21swapper/323:07:453
3265499270,11cyclictest0-21swapper/322:53:413
3265499270,11cyclictest0-21swapper/320:01:023
3264299271,10cyclictest4348-21hddtemp_smartct00:10:171
3263499272,10cyclictest25512-21kworker/0:121:18:530
3265499261,9cyclictest0-21swapper/323:45:573
3265499261,9cyclictest0-21swapper/320:31:133
3265499261,9cyclictest0-21swapper/319:48:383
3265499261,12cyclictest0-21swapper/321:47:033
3265499261,12cyclictest0-21swapper/321:24:173
3265499261,11cyclictest0-21swapper/321:06:473
3265499261,10cyclictest0-21swapper/319:21:323
3265499260,9cyclictest0-21swapper/323:27:453
3265499260,9cyclictest0-21swapper/323:27:453
3265499260,19cyclictest0-21swapper/319:15:493
3265499260,12cyclictest0-21swapper/323:03:533
3265499260,12cyclictest0-21swapper/322:23:543
3265499260,12cyclictest0-21swapper/321:52:533
3265499260,11cyclictest0-21swapper/321:00:573
3265499260,11cyclictest0-21swapper/300:13:103
3265499260,10cyclictest0-21swapper/323:40:293
3264299262,4cyclictest18180-21munin-run23:30:021
3264299262,4cyclictest18180-21munin-run23:30:021
3264299261,6cyclictest0-21swapper/123:50:241
3265499251,8cyclictest0-21swapper/320:23:133
32654992511,4cyclictest0-21swapper/322:14:183
3265499250,7cyclictest0-21swapper/323:18:173
3265499250,11cyclictest0-21swapper/321:13:293
3265499250,11cyclictest0-21swapper/300:00:383
3265499250,10cyclictest0-21swapper/322:43:223
3265499250,10cyclictest0-21swapper/322:05:543
3264299251,6cyclictest0-21swapper/121:34:111
3264299251,16cyclictest0-21swapper/120:00:191
3263499252,9cyclictest17490-21kworker/0:222:15:410
3263499252,9cyclictest17490-21kworker/0:222:15:410
3263499251,5cyclictest0-21swapper/022:34:460
3263499251,10cyclictest29044-21kworker/0:100:08:170
306632250,4sleep23264899cyclictest21:30:162
300622250,1sleep10-21swapper/122:40:251
215322250,4sleep230-21rcuc/219:55:162
215322250,4sleep230-21rcuc/219:55:162
3265499241,9cyclictest0-21swapper/323:22:153
3265499241,9cyclictest0-21swapper/319:50:143
3265499241,9cyclictest0-21swapper/300:25:263
3265499241,11cyclictest0-21swapper/300:18:383
3265499241,10cyclictest0-21swapper/321:31:133
3265499241,10cyclictest0-21swapper/320:59:373
3265499240,11cyclictest0-21swapper/300:33:423
3264299242,10cyclictest0-21swapper/100:32:081
32642992411,10cyclictest32563-21/usr/sbin/munin21:35:131
3264299240,6cyclictest0-21swapper/121:40:181
3264299240,10cyclictest21340-21diskmemload21:45:071
3263499243,8cyclictest31392-40dnf21:30:250
3265499231,11cyclictest0-21swapper/300:06:283
3265499230,10cyclictest0-21swapper/320:35:333
3264899236,5cyclictest0-21swapper/200:00:182
3264899231,11cyclictest0-21swapper/200:13:002
3264299232,16cyclictest0-21swapper/100:28:221
32642992315,3cyclictest0-21swapper/120:36:431
32642992314,4cyclictest0-21swapper/122:46:471
3264299231,19cyclictest619-21cat19:10:131
3264299231,16cyclictest0-21swapper/121:24:541
3264299231,16cyclictest0-21swapper/119:50:061
3263499230,11cyclictest11603-21idleruntime-cro23:15:000
3264299224,9cyclictest0-21swapper/123:30:201
3264299222,18cyclictest0-21swapper/119:15:561
3264299221,19cyclictest0-21swapper/121:15:021
3264299221,15cyclictest0-21swapper/120:44:251
3264299221,15cyclictest0-21swapper/120:23:381
3264299221,12cyclictest25940-21sed21:20:021
3264299221,12cyclictest0-21swapper/123:43:091
3264299221,11cyclictest27900-21diskstats20:10:141
3264299221,10cyclictest0-21swapper/123:08:181
32642992210,3cyclictest26747-21cpuspeed_turbos22:35:131
3264299220,16cyclictest0-21swapper/120:47:021
3263499221,4cyclictest0-21swapper/022:52:570
3265499216,3cyclictest0-21swapper/322:46:273
3264899213,10cyclictest790-21in:imjournal20:30:012
3264899212,16cyclictest2345-21tail22:50:282
3264299212,14cyclictest87550irq/128-enp1s0-22:58:571
32642992116,3cyclictest8919-21proc_pri19:25:221
3264299211,3cyclictest27013-21sed23:50:011
3264299211,3cyclictest0-21swapper/122:09:561
32642992112,3cyclictest0-21swapper/121:08:191
3264299211,17cyclictest21340-21diskmemload22:27:141
3264299211,15cyclictest0-21swapper/120:19:021
3264299211,15cyclictest0-21swapper/120:09:391
3264299211,15cyclictest0-21swapper/100:07:401
3264299211,14cyclictest0-21swapper/119:46:041
3264299211,14cyclictest0-21swapper/100:37:421
3264299211,11cyclictest0-21swapper/123:38:321
3264299211,11cyclictest0-21swapper/123:22:561
3264299211,10cyclictest0-21swapper/123:04:401
3264299210,15cyclictest0-21swapper/123:59:361
3263499219,3cyclictest28935-21kworker/0:020:13:100
3263499217,2cyclictest32481-21kworker/0:019:25:150
3263499212,11cyclictest12454-21/usr/sbin/munin00:30:130
32634992110,3cyclictest5963-21awk20:35:020
3264899205,5cyclictest0-21swapper/223:20:092
3264899205,3cyclictest12050irq/123-ahci[0023:00:012
3264899203,9cyclictest0-21swapper/221:43:272
3264899203,4cyclictest8564-1kworker/u9:100:34:342
3264899202,6cyclictest15374-21munin-run22:10:012
3264899200,9cyclictest0-21swapper/222:41:512
3264299208,2cyclictest19991-21/usr/sbin/munin22:20:161
3264299202,16cyclictest0-21swapper/123:14:261
3264299201,8cyclictest9126-21seq00:20:211
32642992015,4cyclictest0-21swapper/119:30:271
32642992012,2cyclictest0-21swapper/120:26:101
3264299201,14cyclictest0-21swapper/120:57:261
3264299201,11cyclictest13780-21latency_hist23:20:021
3264299201,11cyclictest0-21swapper/122:04:521
3264299201,10cyclictest7140-21df21:50:161
3264299201,10cyclictest31392-21kworker/u8:200:02:311
3264299200,18cyclictest0-21swapper/122:50:191
3264299200,14cyclictest0-21swapper/120:32:491
3263499209,3cyclictest3378-21/usr/sbin/munin19:15:180
3263499208,5cyclictest14909-21cstates00:35:140
3263499206,7cyclictest11-21rcuc/021:25:010
3263499206,3cyclictest18537-21kworker/0:120:00:160
3263499206,2cyclictest2673-21kworker/0:322:55:240
3263499204,7cyclictest12281-21kworker/0:020:53:220
32634992014,2cyclictest17663-21kworker/0:121:05:150
3263499201,2cyclictest0-21swapper/023:05:230
3263499201,11cyclictest20607-21/usr/sbin/munin23:35:220
55562191,2sleep05557-21sshd00:14:080
3264899194,3cyclictest87350irq/126-enp1s0-20:40:012
3264899194,3cyclictest0-21swapper/220:30:112
3264899193,3cyclictest648-21fschecks_time21:35:172
3264899192,5cyclictest1-21systemd21:20:092
3264899192,3cyclictest17543-21needreboot19:45:182
3264899191,7cyclictest0-21swapper/222:18:002
3264899191,7cyclictest0-21swapper/222:18:002
3264899191,7cyclictest0-21swapper/200:25:122
3264899191,5cyclictest10614-21needreboot23:10:212
32648991914,3cyclictest27173-21grep22:35:162
32648991914,3cyclictest13194-21sort20:50:152
3264899191,3cyclictest0-21swapper/223:45:242
32648991913,4cyclictest5642-21ntpq21:45:222
3264899191,11cyclictest0-21swapper/223:54:472
3264899191,10cyclictest16280-21taskset00:35:492
3264299196,4cyclictest6207-21df_inode00:15:131
32642991913,3cyclictest225492sleep119:55:271
32642991913,3cyclictest225492sleep119:55:271
3264299191,16cyclictest0-21swapper/122:11:011
3264299191,16cyclictest0-21swapper/119:20:421
32642991911,3cyclictest0-21swapper/120:52:121
3264299191,10cyclictest17514-21diskstats21:00:151
3264299191,10cyclictest15427-21ntp_states19:40:231
3263499198,3cyclictest22808-21/usr/sbin/munin23:40:160
3263499197,3cyclictest20068-21kworker/0:023:31:230
3263499193,14cyclictest6586-21awk21:50:010
3263499191,6cyclictest0-21swapper/022:41:060
3263499191,5cyclictest17567-21munin-run22:15:000
3263499191,4cyclictest0-21swapper/000:04:050
32634991914,3cyclictest10441-21sed21:55:240
3263499191,3cyclictest3772-21basename20:30:020
32634991913,3cyclictest15609-21tail20:55:170
3263499191,10cyclictest808-21snmpd20:24:330
3263499191,10cyclictest28935-21kworker/0:020:35:160
3263499191,10cyclictest2408-21/usr/sbin/munin21:40:200
91362181,2sleep20-21swapper/223:06:522
3264899187,3cyclictest87350irq/126-enp1s0-21:10:152
3264899185,3cyclictest0-21swapper/223:59:562
3264899183,8cyclictest87350irq/126-enp1s0-23:40:142
3264899183,8cyclictest0-21swapper/222:04:402
3264899182,1cyclictest0-21swapper/200:20:142
3264899182,13cyclictest2078-21grep00:05:182
3264899181,7cyclictest18521-21cpuspeed_turbos23:30:142
3264899181,15cyclictest0-21swapper/223:27:012
3264899181,15cyclictest0-21swapper/223:27:002
3264899181,11cyclictest0-21swapper/221:55:452
3264299187,9cyclictest29864-21kworker/u8:121:55:071
32642991814,3cyclictest0-21swapper/122:34:131
3264299181,3cyclictest13022-21cat19:35:181
3264299181,16cyclictest0-21swapper/121:28:231
3263499186,3cyclictest586-21cat22:50:020
3263499182,10cyclictest17663-21kworker/0:121:04:150
3263499181,9cyclictest808-21snmpd00:26:290
3263499181,5cyclictest8564-1kworker/u9:121:11:050
32634991813,3cyclictest26136-21pmu-power23:45:210
3263499181,10cyclictest13400-21/usr/sbin/munin22:05:210
173822180,2sleep20-21swapper/222:12:512
3264899172,5cyclictest0-21swapper/222:29:122
3264899171,9cyclictest21340-21diskmemload21:15:282
3264899171,6cyclictest25366-21/usr/sbin/munin22:30:202
3264899171,3cyclictest0-21swapper/219:40:002
32648991712,3cyclictest565-21ntp_states20:20:232
32648991712,3cyclictest28081-21grep20:10:152
32648991712,3cyclictest20945-21df23:35:142
3264899171,15cyclictest0-21swapper/220:17:152
3264899171,15cyclictest0-21swapper/219:43:062
32648991711,4cyclictest18207-21grep21:00:212
3263499174,4cyclictest9371-21kworker/0:219:38:190
3263499172,10cyclictest19720-21kworker/0:322:38:120
3263499171,7cyclictest0-21swapper/021:35:060
3263499171,6cyclictest0-21swapper/023:50:450
3263499171,6cyclictest0-21swapper/022:04:020
32634991713,2cyclictest30495-21open_inodes23:55:230
3263499171,2cyclictest6785-21kworker/0:021:50:190
32634991712,3cyclictest9592-21grep00:20:260
32634991712,3cyclictest5912-21sh19:20:110
32634991712,3cyclictest29154-21strings21:25:190
32634991712,3cyclictest12613-21sed20:50:010
3263499171,10cyclictest0-21swapper/023:29:380
3263499171,10cyclictest0-21swapper/023:29:380
316632171,2sleep00-21swapper/020:19:320
3264899164,3cyclictest19428-21smartctl19:50:162
3264899163,3cyclictest5750irq/9-acpi23:00:202
3264899162,6cyclictest0-21swapper/222:20:232
3264899161,7cyclictest12405-21gltestperf23:15:162
3264899161,6cyclictest23148-21/usr/sbin/munin20:00:142
3264899161,4cyclictest0-21swapper/221:25:332
3264899161,2cyclictest0-21swapper/219:13:402
32648991612,3cyclictest0-21swapper/219:19:522
3264899161,14cyclictest0-21swapper/200:17:002
32648991611,3cyclictest6917-21expr21:50:162
32648991611,3cyclictest10942-21grep20:45:152
3263499165,3cyclictest10102-21cpuspeed_turbos19:30:130
3263499165,2cyclictest87550irq/128-enp1s0-00:16:150
3263499161,12cyclictest8378-21/usr/sbin/munin20:40:190
3264899159,3cyclictest8912-21fschecks_time20:40:152
3264899158,4cyclictest0-21swapper/220:55:232
3264899152,4cyclictest25768-21grep20:05:152
32648991512,2cyclictest0-21swapper/221:05:552
32648991510,3cyclictest8501-21awk19:25:182
32648991510,3cyclictest5776-21expr19:20:112
32648991510,3cyclictest10195-21expr19:30:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional