You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-29 - 08:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Mon Dec 29, 2025 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
79132236162,23sleep10-21swapper/119:09:531
77902202166,24sleep20-21swapper/219:08:172
59592201166,23sleep30-21swapper/319:05:043
59662200162,26sleep00-21swapper/019:05:090
1399221820,4sleep1800999cyclictest19:20:151
1137921360,3sleep0800299cyclictest00:10:020
167482920,4sleep3802199cyclictest19:25:243
211482660,8sleep341-21ksoftirqd/319:35:233
250342530,7sleep011-21rcuc/022:11:480
302902410,1sleep00-21swapper/022:25:160
802199345,12cyclictest0-21swapper/323:15:093
802199341,12cyclictest8851-21memory21:35:203
802199338,4cyclictest801-21snmpd22:25:433
802199321,14cyclictest6034-21gzip23:55:233
802199311,14cyclictest0-21swapper/321:52:043
800299302,13cyclictest20906-21munin-run22:05:010
800299295,11cyclictest3936-21awk22:40:020
8021992812,3cyclictest17299-21perl21:55:163
802199281,14cyclictest0-21swapper/319:50:273
800999281,18cyclictest23100-21hddtemp_smartct00:35:201
802199279,12cyclictest0-21swapper/320:38:473
802199271,22cyclictest31408-21fschecks_time20:00:153
8021992712,10cyclictest0-21swapper/321:04:573
802199271,13cyclictest764-21ntpd23:25:213
802199271,13cyclictest0-21swapper/300:10:193
802199270,18cyclictest0-21swapper/320:10:113
802199270,18cyclictest0-21swapper/320:10:113
800299271,13cyclictest7704-21munin-run21:35:010
802199269,3cyclictest0-21swapper/321:41:083
802199261,13cyclictest0-21swapper/323:12:033
802199261,13cyclictest0-21swapper/319:10:513
802199261,12cyclictest0-21swapper/323:38:203
802199261,12cyclictest0-21swapper/320:53:473
802199261,12cyclictest0-21swapper/319:33:483
802199261,11cyclictest0-21swapper/321:07:553
802199261,11cyclictest0-21swapper/320:08:353
8002992610,14cyclictest404-21perf20:05:010
800299260,13cyclictest11497-21awk20:30:020
802199259,3cyclictest0-21swapper/322:30:573
802199251,19cyclictest23148-21ntp_states19:40:243
802199251,11cyclictest0-21swapper/323:31:243
802199251,11cyclictest0-21swapper/323:03:313
802199251,11cyclictest0-21swapper/319:58:353
802199251,10cyclictest86950irq/126-enp1s0-00:18:493
802199251,10cyclictest0-21swapper/323:08:513
204212250,4sleep320424-21unixbench_multi22:00:263
802199248,3cyclictest0-21swapper/320:45:393
802199246,3cyclictest0-21swapper/322:15:253
802199241,5cyclictest31402-21kernelversion23:40:203
802199241,21cyclictest0-21swapper/300:28:233
802199241,20cyclictest0-21swapper/300:23:253
802199241,15cyclictest0-21swapper/319:16:033
802199241,12cyclictest8092-21chrt22:47:133
802199241,11cyclictest0-21swapper/323:50:433
802199241,11cyclictest0-21swapper/321:19:473
802199241,10cyclictest13972-21hddtemp_smartct19:20:163
802199241,10cyclictest0-21swapper/322:43:473
800999240,19cyclictest0-21swapper/100:24:061
800299246,15cyclictest17127-21munin-run23:10:010
802199238,3cyclictest0-21swapper/322:12:593
802199236,3cyclictest0-21swapper/320:59:073
802199234,3cyclictest0-21swapper/322:24:593
802199231,9cyclictest0-21swapper/322:37:313
8021992311,3cyclictest0-21swapper/323:20:213
802199231,10cyclictest0-21swapper/322:59:473
802199231,10cyclictest0-21swapper/322:53:553
802199231,10cyclictest0-21swapper/300:00:353
8016992315,5cyclictest20715-21systemd-journal22:02:472
800999236,6cyclictest0-21swapper/121:56:041
8009992314,2cyclictest0-21swapper/121:15:201
8009992313,2cyclictest0-21swapper/100:10:191
800999231,16cyclictest0-21swapper/100:31:241
800999231,16cyclictest0-21swapper/100:04:021
800999231,13cyclictest741-21ntp_states22:30:211
800999231,11cyclictest0-21swapper/122:52:411
800999231,11cyclictest0-21swapper/122:00:301
800299235,15cyclictest19314-21idleruntime-cro23:15:010
800299234,16cyclictest26268-21idleruntime-cro19:50:000
800299234,11cyclictest0-21swapper/023:50:280
253092230,4sleep2801699cyclictest23:25:252
802199228,3cyclictest0-21swapper/323:46:193
802199228,3cyclictest0-21swapper/323:46:193
802199227,3cyclictest0-21swapper/320:40:273
802199225,3cyclictest0-21swapper/321:11:073
802199225,3cyclictest0-21swapper/300:31:433
802199221,14cyclictest0-21swapper/320:16:033
802199221,13cyclictest0-21swapper/320:30:113
802199221,12cyclictest0-21swapper/320:24:353
802199221,11cyclictest0-21swapper/321:23:233
802199221,10cyclictest0-21swapper/322:06:353
802199221,10cyclictest0-21swapper/321:26:203
802199221,10cyclictest0-21swapper/321:26:203
8016992213,6cyclictest12544-21systemd-journal22:57:582
800999229,4cyclictest0-21swapper/100:25:191
800999229,11cyclictest8472-21grep21:35:161
800999223,11cyclictest27323-21latency19:50:181
800999222,12cyclictest0-21swapper/121:50:261
800999221,13cyclictest23107-21perf22:10:001
800999220,4cyclictest0-21swapper/123:52:421
800299221,14cyclictest0-21swapper/023:15:250
163412220,2sleep00-21swapper/000:20:180
802199218,3cyclictest0-21swapper/319:45:083
802199217,6cyclictest22960-21uniq00:35:183
802199217,3cyclictest0-21swapper/321:34:523
802199211,13cyclictest0-21swapper/320:28:353
800999212,11cyclictest87150irq/128-enp1s0-23:55:221
800999212,10cyclictest29076-21munin-run21:10:011
800999211,9cyclictest0-21swapper/123:14:041
800999211,3cyclictest1497-21pluginstate23:45:251
800999211,3cyclictest1497-21pluginstate23:45:251
8009992113,3cyclictest0-21swapper/123:33:351
8009992113,2cyclictest0-21swapper/123:02:231
800999211,18cyclictest0-21swapper/120:23:211
800999211,12cyclictest0-21swapper/122:41:051
800999211,11cyclictest6696-21diskstats22:45:161
800999211,10cyclictest0-21swapper/121:27:141
800999211,10cyclictest0-21swapper/121:27:141
8009992110,5cyclictest914-21kworker/u8:221:20:141
8009992110,3cyclictest17688-21users20:40:261
800999210,16cyclictest0-21swapper/123:07:071
800999210,16cyclictest0-21swapper/122:58:271
800999210,16cyclictest0-21swapper/100:18:521
800299214,9cyclictest0-21swapper/019:50:420
800299212,12cyclictest0-21swapper/023:22:240
800299211,9cyclictest0-21swapper/021:25:180
800299211,9cyclictest0-21swapper/021:25:180
8002992116,3cyclictest18395-21grep00:25:150
800299211,13cyclictest0-21swapper/021:49:420
8002992110,3cyclictest31501-21/usr/sbin/munin21:15:160
57022210,1sleep20-21swapper/223:55:192
802199201,11cyclictest0-21swapper/300:06:123
802199201,10cyclictest0-21swapper/321:45:573
800999209,3cyclictest25583-21hostname21:00:201
8009992016,3cyclictest0-21swapper/120:59:411
8009992014,4cyclictest25233-21runrttasks22:14:211
800999201,14cyclictest0-21swapper/123:36:561
800999201,14cyclictest0-21swapper/123:27:451
800999201,14cyclictest0-21swapper/122:27:131
800999201,14cyclictest0-21swapper/122:19:481
800999201,14cyclictest0-21swapper/121:11:021
800999201,13cyclictest0-21swapper/123:40:211
800999201,13cyclictest0-21swapper/121:41:321
8009992011,3cyclictest0-21swapper/100:05:141
800999201,11cyclictest22091-21diskstats23:20:151
800999201,11cyclictest19699-21cat19:35:011
800999201,11cyclictest11881-21if_err_enp1s019:15:181
800999201,10cyclictest16885-21sed19:25:241
8009992010,4cyclictest21483-21kworker/u8:120:18:001
800299207,3cyclictest27730-21users23:30:290
800299201,7cyclictest0-21swapper/000:38:240
800299201,6cyclictest0-21swapper/022:50:540
800299201,5cyclictest0-21swapper/022:46:370
800299201,4cyclictest0-21swapper/000:32:300
800299201,3cyclictest0-21swapper/023:27:310
800299201,16cyclictest24686-21idleruntime-cro21:00:000
800299201,11cyclictest826-21lldpd19:15:030
800299201,11cyclictest32113-21/usr/sbin/munin22:30:120
800299201,11cyclictest22086-21/usr/sbin/munin19:40:270
801699196,3cyclictest0-21swapper/221:18:122
801699191,8cyclictest0-21swapper/223:45:222
801699191,8cyclictest0-21swapper/223:45:222
801699191,4cyclictest0-21swapper/221:27:112
801699191,4cyclictest0-21swapper/221:27:102
8016991914,3cyclictest29685-21idleruntime-cro22:25:002
8016991913,3cyclictest30782-21sed21:10:282
8016991913,3cyclictest18649-21diskstats20:45:152
8016991912,4cyclictest29351-21/usr/sbin/munin19:55:172
801699191,16cyclictest0-21swapper/222:25:132
801699191,12cyclictest0-21swapper/221:55:242
800999199,3cyclictest0-21swapper/120:07:271
800999196,5cyclictest0-21swapper/122:20:291
800999191,9cyclictest26374-21latency_hist19:50:011
8009991915,3cyclictest0-21swapper/119:14:531
800999191,17cyclictest0-21swapper/123:19:421
800999191,17cyclictest0-21swapper/120:14:281
800999191,17cyclictest0-21swapper/120:14:281
800999191,17cyclictest0-21swapper/119:35:531
800999191,11cyclictest0-21swapper/121:45:241
800999191,10cyclictest5991-21expr21:30:131
800999191,10cyclictest14187-21cstates20:35:151
800999191,10cyclictest0-21swapper/119:57:561
800299198,3cyclictest4700-21sshd20:12:440
800299198,3cyclictest4700-21sshd20:12:440
800299192,9cyclictest0-21swapper/000:15:440
8002991913,3cyclictest19287-21needreboot20:45:200
800299191,13cyclictest0-21swapper/021:37:250
800299191,12cyclictest0-21swapper/022:19:160
800299191,12cyclictest0-21swapper/020:50:170
800299191,10cyclictest31488-21memory23:40:190
800299191,10cyclictest29500-21sshd22:22:430
800299191,10cyclictest22390-21smart_sda22:05:240
800299191,10cyclictest14526-21/usr/sbin/munin21:50:150
800299190,11cyclictest22350-21kworker/0:021:00:240
163752191,2sleep20-21swapper/221:53:402
801699181,7cyclictest0-21swapper/221:33:322
801699181,6cyclictest0-21swapper/200:10:092
801699181,5cyclictest0-21swapper/221:45:092
801699181,4cyclictest0-21swapper/222:07:232
8016991812,3cyclictest8487-21proc_pri20:20:252
8016991812,3cyclictest10527-21idleruntime-cro22:55:012
8009991814,3cyclictest0-21swapper/120:03:071
800999181,2cyclictest0-21swapper/122:35:231
800999181,16cyclictest0-21swapper/119:43:471
800999181,15cyclictest0-21swapper/120:45:081
800999180,17cyclictest0-21swapper/120:53:231
800299187,3cyclictest28676-21/usr/sbin/munin19:55:160
800299181,9cyclictest0-21swapper/021:40:190
800299181,8cyclictest0-21swapper/023:47:080
800299181,8cyclictest0-21swapper/023:47:070
800299181,7cyclictest0-21swapper/021:10:460
800299181,6cyclictest0-21swapper/000:00:110
800299181,4cyclictest0-21swapper/023:03:130
800299181,3cyclictest14824-21memory20:35:210
8002991813,3cyclictest5719-21/usr/sbin/munin23:55:200
8002991813,3cyclictest12950-21tail00:10:280
800299181,12cyclictest0-21swapper/021:24:250
111632181,2sleep10-21swapper/120:26:021
801699176,3cyclictest12050irq/123-ahci[0020:30:012
801699172,3cyclictest87250irq/129-enp1s0-21:05:232
801699171,9cyclictest0-21swapper/200:20:112
801699171,7cyclictest0-21swapper/223:08:392
801699171,6cyclictest0-21swapper/221:39:002
801699171,5cyclictest8068-21ntpq00:00:202
801699171,5cyclictest0-21swapper/221:20:542
801699171,4cyclictest0-21swapper/222:31:012
801699171,3cyclictest0-21swapper/223:51:092
8016991712,4cyclictest21220-21kworker/u8:123:33:542
8016991712,3cyclictest27487-21latency_hist22:20:012
8016991712,3cyclictest24655-21fschecks_count19:45:152
8016991712,3cyclictest16444-21wc19:25:192
801699171,15cyclictest0-21swapper/200:27:502
8016991711,4cyclictest0-21swapper/223:00:172
8016991711,3cyclictest20494-21/usr/sbin/munin20:50:222
801699171,10cyclictest0-21swapper/222:36:212
8016991710,4cyclictest23273-21/usr/sbin/munin00:35:212
801699170,6cyclictest0-21swapper/220:10:282
801699170,6cyclictest0-21swapper/220:10:282
800999170,4cyclictest0-21swapper/120:32:541
800299173,3cyclictest0-21swapper/019:27:570
800299171,8cyclictest0-21swapper/022:58:190
800299171,7cyclictest19883-21/usr/sbin/munin19:35:210
800299171,5cyclictest0-21swapper/023:37:560
8002991713,2cyclictest11-21rcuc/019:20:150
8002991712,3cyclictest18425-21if_enp1s019:30:160
800299171,15cyclictest0-21swapper/020:34:240
800299171,13cyclictest4150-21/usr/sbin/munin22:40:240
800299171,13cyclictest27110-21/usr/sbin/munin21:05:220
8002991711,3cyclictest2688-21idleruntime-cro20:10:020
57572171,2sleep20-21swapper/222:40:312
233902170,2sleep20-21swapper/223:21:012
155372170,2sleep20-21swapper/200:19:192
801699161,7cyclictest20232-21hwlatdetect23:15:172
801699161,6cyclictest30875-21/usr/sbin/munin20:00:172
801699161,6cyclictest26864-21fschecks_count19:50:142
801699161,6cyclictest10470-21df21:40:132
801699161,6cyclictest0-21swapper/223:40:222
801699161,6cyclictest0-21swapper/222:47:002
801699161,6cyclictest0-21swapper/219:35:132
801699161,5cyclictest0-21swapper/222:10:192
801699161,4cyclictest0-21swapper/220:40:172
8016991612,3cyclictest0-21swapper/219:13:242
8016991612,3cyclictest0-21swapper/200:05:122
801699161,14cyclictest0-21swapper/223:39:192
801699161,14cyclictest0-21swapper/221:00:382
801699161,14cyclictest0-21swapper/219:23:542
8016991611,3cyclictest22899-21expr20:55:142
8016991611,3cyclictest20223-21/usr/sbin/munin00:30:182
8016991611,3cyclictest11029-21grep19:15:052
8016991610,4cyclictest17460-21cpuspeed_turbos23:10:142
8016991610,4cyclictest0-21swapper/219:30:122
800299162,3cyclictest0-21swapper/019:12:060
800299161,3cyclictest0-21swapper/021:55:310
8002991612,3cyclictest0-21swapper/020:40:540
800299161,14cyclictest0-21swapper/020:24:380
8002991611,3cyclictest5432-21expr20:15:160
801699159,4cyclictest12130-21cut20:30:142
801699151,5cyclictest0-21swapper/220:15:182
8016991510,3cyclictest927-21cat20:05:132
8016991510,3cyclictest22093-21basename19:40:142
801699150,14cyclictest0-21swapper/220:35:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional