You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-02 - 04:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sun Aug 02, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8172237160,24sleep00-21swapper/019:09:300
8592236163,24sleep20-21swapper/219:09:592
5482230156,23sleep10-21swapper/119:06:451
6032203168,23sleep30-21swapper/319:07:243
968999693,2cyclictest0-21swapper/219:44:392
189082920,1sleep20-21swapper/221:00:202
11732760,3sleep10-21swapper/100:00:261
240352560,2sleep20-21swapper/222:25:242
215432560,4sleep121-21rcuc/119:55:011
189152560,4sleep232-21ksoftirqd/223:30:012
293382550,3sleep324702-21kworker/3:222:40:023
147482510,1sleep30-21swapper/323:20:153
68652500,3sleep30-21swapper/319:20:163
958994010,14cyclictest14839-21munin-run19:40:010
95899349,11cyclictest29279-21idleruntime-cro22:40:010
97599327,5cyclictest0-21swapper/320:56:383
241532300,2sleep397599cyclictest22:25:253
95899291,13cyclictest6051-21munin-run19:20:000
97599279,4cyclictest0-21swapper/323:45:543
97599271,15cyclictest0-21swapper/322:45:233
97599270,9cyclictest0-21swapper/320:25:243
97599261,18cyclictest0-21swapper/300:05:223
97599261,14cyclictest0-21swapper/319:42:113
96899267,9cyclictest0-21swapper/222:20:012
96899261,6cyclictest0-21swapper/222:37:372
95899267,16cyclictest32094-21awk00:00:010
97599254,11cyclictest1250-21/usr/sbin/munin00:00:263
97599251,13cyclictest0-21swapper/321:05:193
13242250,1sleep00-21swapper/020:20:210
97599241,6cyclictest0-21swapper/322:31:083
97599241,6cyclictest0-21swapper/322:14:273
95899246,3cyclictest26394-21hddtemp_smartct23:45:170
95899242,11cyclictest2499-21/usr/sbin/munin20:25:170
95899242,10cyclictest8138-21/usr/sbin/munin23:05:170
95899241,11cyclictest19129-21/usr/sbin/munin23:30:160
45632240,2sleep20-21swapper/220:30:022
161582241,18sleep216211-21latency00:35:202
97599237,3cyclictest0-21swapper/323:07:233
97599237,11cyclictest0-21swapper/322:57:033
97599231,6cyclictest0-21swapper/322:52:173
97599231,6cyclictest0-21swapper/322:03:313
97599231,6cyclictest0-21swapper/321:55:123
97599230,12cyclictest0-21swapper/319:31:553
96899231,6cyclictest16338-21vmstat23:20:302
96899231,6cyclictest0-21swapper/223:51:422
96899231,6cyclictest0-21swapper/222:22:052
96899231,6cyclictest0-21swapper/222:22:052
96899231,6cyclictest0-21swapper/222:01:012
963992316,4cyclictest5624-21/usr/sbin/munin00:10:221
96399231,19cyclictest0-21swapper/123:50:121
95899237,4cyclictest0-21swapper/022:44:080
95899236,3cyclictest16929-21/usr/sbin/munin23:25:190
95899232,11cyclictest7512-21/usr/sbin/munin21:50:110
958992317,3cyclictest13779-21cstates20:50:130
95899231,5cyclictest0-21swapper/000:27:420
95899231,3cyclictest0-21swapper/021:45:130
95899231,11cyclictest21658-21/usr/sbin/munin19:55:180
95899231,11cyclictest13655-21cut23:15:240
95899231,10cyclictest3720-21sshd22:55:030
95899231,10cyclictest0-21swapper/021:12:030
95899231,10cyclictest0-21swapper/021:12:020
249602231,2sleep00-21swapper/020:00:230
249602231,2sleep00-21swapper/020:00:230
97599229,3cyclictest0-21swapper/319:16:553
97599227,3cyclictest0-21swapper/321:29:113
97599226,3cyclictest0-21swapper/300:19:543
97599221,7cyclictest0-21swapper/323:51:523
97599221,6cyclictest0-21swapper/323:03:373
97599221,6cyclictest0-21swapper/321:34:133
97599221,6cyclictest0-21swapper/321:12:443
97599221,6cyclictest0-21swapper/321:12:443
97599221,5cyclictest0-21swapper/321:15:173
97599221,4cyclictest20244-21kworker/u8:122:20:193
97599221,4cyclictest20244-21kworker/u8:122:20:193
97599221,4cyclictest0-21swapper/300:21:363
97599221,2cyclictest4145-21kworker/u8:121:20:153
96899221,6cyclictest0-21swapper/222:51:452
96899221,6cyclictest0-21swapper/221:53:162
96899221,6cyclictest0-21swapper/221:35:512
96899221,6cyclictest0-21swapper/200:18:192
96899221,5cyclictest31183-21ntp_states23:55:232
96899221,5cyclictest0-21swapper/221:21:082
96899221,5cyclictest0-21swapper/221:13:332
96899221,5cyclictest0-21swapper/221:13:332
96899221,4cyclictest0-21swapper/223:36:162
96399223,16cyclictest16799-21systemd-journal19:42:091
96399222,9cyclictest23-21ksoftirqd/122:59:081
96399221,4cyclictest1847-21expr22:50:141
96399221,3cyclictest25553-21ntp_states21:15:231
96399221,18cyclictest126262sleep119:34:591
96399220,4cyclictest0-21swapper/122:27:341
95899227,3cyclictest9709-21/usr/sbin/munin21:55:200
95899226,3cyclictest87550irq/128-enp1s0-20:30:280
95899225,4cyclictest14986-21runrttasks23:20:160
95899223,4cyclictest1121-21sshd22:48:030
95899222,8cyclictest24490-21/usr/sbin/munin21:15:120
95899221,8cyclictest18072-21sshd22:12:290
95899221,10cyclictest29643-21sshd23:52:030
95899221,10cyclictest0-21swapper/000:09:410
95899220,14cyclictest0-21swapper/023:04:030
97599215,10cyclictest0-21swapper/321:49:033
97599212,10cyclictest3795-21hddtemp_smartct21:40:153
97599210,9cyclictest0-21swapper/319:56:513
96899218,6cyclictest0-21swapper/200:03:052
96899211,7cyclictest0-21swapper/222:48:352
96899211,6cyclictest0-21swapper/221:56:152
96899211,6cyclictest0-21swapper/221:41:192
96899211,6cyclictest0-21swapper/221:31:192
96899211,6cyclictest0-21swapper/221:27:572
96899211,5cyclictest0-21swapper/223:04:032
96899211,5cyclictest0-21swapper/222:56:082
96899211,5cyclictest0-21swapper/222:11:072
96899211,5cyclictest0-21swapper/221:18:452
96399212,17cyclictest0-21swapper/120:40:301
96399212,16cyclictest592-21munin-run21:35:011
963992114,2cyclictest0-21swapper/119:50:011
96399211,11cyclictest7445-21perl20:35:141
96399210,20cyclictest0-21swapper/123:48:241
95899219,3cyclictest32629-21head21:30:230
95899213,7cyclictest0-21swapper/000:24:200
95899211,9cyclictest0-21swapper/021:28:060
95899211,8cyclictest844-21ls00:00:210
958992116,3cyclictest3229-21cut21:40:120
95899211,17cyclictest0-21swapper/022:06:430
95899211,10cyclictest0-21swapper/021:05:270
122812211,3sleep00-21swapper/019:30:330
97599209,8cyclictest827-21runrttasks23:55:253
97599206,3cyclictest0-21swapper/320:07:143
97599205,5cyclictest0-21swapper/323:20:013
97599205,3cyclictest0-21swapper/320:40:223
97599204,11cyclictest0-21swapper/300:12:063
97599202,6cyclictest18245-1kworker/u9:221:35:493
97599202,11cyclictest9345-21latency19:25:183
97599201,9cyclictest0-21swapper/320:45:343
97599201,5cyclictest0-21swapper/300:31:593
97599201,4cyclictest18245-1kworker/u9:223:37:113
97599201,4cyclictest0-21swapper/300:27:163
97599201,3cyclictest4698-21/usr/sbin/munin20:30:263
97599201,3cyclictest0-21swapper/323:44:473
97599201,3cyclictest0-21swapper/322:09:123
96899202,5cyclictest18245-1kworker/u9:200:14:552
96899201,6cyclictest0-21swapper/200:29:162
96899201,5cyclictest18245-1kworker/u9:223:30:372
96899201,5cyclictest18245-1kworker/u9:222:32:342
96899201,5cyclictest18245-1kworker/u9:222:06:282
96899201,5cyclictest0-21swapper/221:46:322
96899201,4cyclictest0-21swapper/200:30:582
96899200,6cyclictest0-21swapper/220:35:152
96399209,3cyclictest0-21swapper/123:18:141
96399207,3cyclictest0-21swapper/122:22:361
96399207,3cyclictest0-21swapper/122:22:361
96399201,5cyclictest6290-21forks23:00:151
963992015,3cyclictest29273-21awk22:40:001
96399201,4cyclictest0-21swapper/121:46:051
96399201,18cyclictest0-21swapper/121:06:211
96399201,18cyclictest0-21swapper/119:18:561
96399201,17cyclictest0-21swapper/120:45:431
96399201,17cyclictest0-21swapper/100:20:131
96399201,11cyclictest28868-21chrt21:25:031
963992010,4cyclictest0-21swapper/122:05:191
96399200,18cyclictest0-21swapper/120:26:181
96399200,18cyclictest0-21swapper/120:15:251
96399200,15cyclictest0-21swapper/100:34:021
96399200,12cyclictest24078-21perl23:40:151
95899201,6cyclictest0-21swapper/023:42:140
95899201,5cyclictest0-21swapper/000:37:240
95899201,17cyclictest0-21swapper/000:10:140
95899201,16cyclictest10446-21munin-run19:30:020
95899201,12cyclictest0-21swapper/021:38:250
95899201,11cyclictest0-21swapper/019:53:000
95899201,10cyclictest0-21swapper/022:04:430
95899201,10cyclictest0-21swapper/000:31:500
97599194,3cyclictest0-21swapper/323:31:553
97599192,3cyclictest0-21swapper/319:36:533
97599191,8cyclictest0-21swapper/320:00:273
97599191,8cyclictest0-21swapper/320:00:273
97599191,5cyclictest18245-1kworker/u9:222:18:433
97599191,3cyclictest0-21swapper/322:40:373
97599191,2cyclictest87350irq/126-enp1s0-19:52:573
97599191,14cyclictest0-21swapper/320:10:363
97599191,10cyclictest30460-21/usr/sbin/munin20:15:153
97599190,5cyclictest32103-21kworker/u8:000:37:413
96899196,4cyclictest22995-21grep19:55:242
96899191,5cyclictest0-21swapper/223:15:052
96899191,5cyclictest0-21swapper/223:06:032
96899191,15cyclictest8580-21grep19:25:122
96399197,9cyclictest8754-21latency_hist00:20:011
96399197,2cyclictest26165-21ntpq22:30:211
96399196,3cyclictest0-21swapper/123:38:561
96399195,4cyclictest0-21swapper/122:45:231
96399195,3cyclictest9489-21sort23:05:261
96399195,11cyclictest0-21swapper/119:35:201
96399194,12cyclictest0-21swapper/123:12:401
96399191,6cyclictest29480-21sshd22:40:031
963991916,2cyclictest0-21swapper/120:05:021
96399191,4cyclictest10622-21memory21:55:191
96399191,4cyclictest0-21swapper/123:34:201
963991914,3cyclictest12232-21cstates22:00:121
96399191,3cyclictest0-21swapper/120:56:401
96399191,16cyclictest0-21swapper/120:30:331
96399191,16cyclictest0-21swapper/120:11:231
96399191,16cyclictest0-21swapper/119:25:031
96399191,15cyclictest17037-21cpuspeed_turbos23:25:121
963991911,3cyclictest0-21swapper/121:40:141
96399191,12cyclictest0-21swapper/122:17:461
96399191,10cyclictest3251-21ntpq00:05:271
96399190,3cyclictest0-21swapper/120:00:461
96399190,3cyclictest0-21swapper/120:00:461
96399190,17cyclictest0-21swapper/121:10:221
96399190,17cyclictest0-21swapper/121:10:221
95899197,11cyclictest0-21swapper/019:50:010
95899195,3cyclictest0-21swapper/022:16:540
95899192,4cyclictest8694-21sshd20:38:030
95899191,6cyclictest0-21swapper/022:25:180
267642191,2sleep00-21swapper/022:31:130
97599187,3cyclictest808-21snmpd19:14:483
97599187,3cyclictest18245-1kworker/u9:221:53:423
97599182,5cyclictest18245-1kworker/u9:223:10:203
97599181,7cyclictest0-21swapper/323:28:043
97599181,10cyclictest839-21lldpd20:24:183
96899186,3cyclictest24242-21fschecks_count20:00:152
96899186,3cyclictest24242-21fschecks_count20:00:142
96899185,2cyclictest87650irq/129-enp1s0-23:41:562
96899181,7cyclictest0-21swapper/223:48:202
96899181,6cyclictest0-21swapper/219:30:592
96899181,5cyclictest0-21swapper/219:15:142
968991814,3cyclictest0-21swapper/200:05:352
968991813,3cyclictest13031-21tr19:35:122
968991813,3cyclictest11961-21hddtemp_smartct20:45:182
968991812,3cyclictest30406-21memory22:40:182
96899181,15cyclictest0-21swapper/220:55:062
96399186,6cyclictest15707-21df_inode00:35:151
96399186,3cyclictest0-21swapper/123:56:141
96399183,12cyclictest11307-21df00:25:131
963991813,3cyclictest7090-21irqstats19:20:181
96399181,2cyclictest0-21swapper/121:20:111
96399181,2cyclictest0-21swapper/119:13:281
96399181,16cyclictest0-21swapper/122:10:121
96399181,16cyclictest0-21swapper/121:36:481
96399181,16cyclictest0-21swapper/119:55:141
96399180,16cyclictest0-21swapper/121:50:361
95899181,9cyclictest0-21swapper/022:22:340
95899181,9cyclictest0-21swapper/022:22:340
95899181,16cyclictest0-21swapper/022:53:130
95899181,11cyclictest0-21swapper/020:59:560
95899181,10cyclictest0-21swapper/019:21:030
958991810,7cyclictest7-21ksoftirqd/023:35:060
144902181,3sleep10-21swapper/123:20:021
97599175,3cyclictest12050irq/123-ahci[0020:40:013
97599174,3cyclictest0-21swapper/319:45:193
975991711,3cyclictest17879-21/usr/sbin/munin21:00:143
96899175,4cyclictest22062-21diskmemload23:11:302
96899171,7cyclictest26359-21cstates20:05:132
96899171,6cyclictest18391-21pmu-power19:45:212
96899171,6cyclictest1105-21kernelversion20:20:182
96899171,6cyclictest0-21swapper/220:30:242
96899171,6cyclictest0-21swapper/219:10:142
96899171,5cyclictest0-21swapper/220:40:242
968991712,3cyclictest30461-21acpi20:15:152
963991711,3cyclictest955-21hwlatdetect20:20:171
963991711,3cyclictest19139-21sensors_temp21:00:231
95899172,13cyclictest6578-21/usr/sbin/munin00:15:130
958991713,3cyclictest0-21swapper/023:10:140
95899171,14cyclictest0-21swapper/020:06:220
95899171,13cyclictest1169-21/usr/sbin/munin19:10:150
144012170,2sleep10-21swapper/120:50:201
975991611,3cyclictest13723-21expr20:50:123
96899164,4cyclictest19452-21kworker/2:020:10:272
96899161,5cyclictest0-21swapper/221:05:142
968991612,3cyclictest0-21swapper/220:50:312
968991611,3cyclictest6898-21gltestperf19:20:162
968991611,3cyclictest20708-21sendmail_mailst19:50:242
968991610,3cyclictest10499-21sshd00:22:032
95899168,5cyclictest28659-21fschecks_count20:10:180
95899168,5cyclictest19854-21crond21:05:010
958991612,3cyclictest0-21swapper/020:41:430
958991612,3cyclictest0-21swapper/020:15:060
958991612,3cyclictest0-21swapper/019:44:020
958991610,3cyclictest27356-21hddtemp_smartct21:20:150
130702161,2sleep00-21swapper/020:47:510
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional