You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-07 - 04:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Fri Aug 07, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
47602232162,58sleep20-21swapper/219:08:032
48722205169,24sleep10-21swapper/119:09:301
48492202166,23sleep30-21swapper/319:09:133
46462202168,23sleep00-21swapper/019:06:350
5013999283,3cyclictest0-21swapper/200:10:002
171732680,3sleep30-21swapper/322:00:233
198942590,3sleep0500299cyclictest20:55:120
321522560,3sleep230-21rcuc/221:20:292
90952520,1sleep20-21swapper/222:55:252
195212510,1sleep00-21swapper/019:40:140
277352470,5sleep011-21rcuc/020:00:010
225842460,1sleep30-21swapper/319:45:243
5013994441,2cyclictest0-21swapper/220:01:352
500299339,11cyclictest24543-21awk22:20:010
500999311,18cyclictest0-21swapper/120:54:401
5002993114,15cyclictest19483-21basename20:55:000
500299303,12cyclictest15690-21munin-run22:00:000
500299301,14cyclictest27348-21munin-run23:40:020
261632300,4sleep121-21rcuc/121:10:011
131652290,4sleep230-21rcuc/223:05:202
501399280,21cyclictest0-21swapper/223:24:442
500299280,23cyclictest8945-21kworker/0:121:47:360
500999263,19cyclictest17077-40dnf22:00:241
5009992622,3cyclictest0-21swapper/122:25:251
501399250,9cyclictest0-21swapper/222:53:442
500999250,6cyclictest0-21swapper/100:11:581
500299250,12cyclictest10072-21awk19:20:020
204312250,5sleep0500299cyclictest22:10:120
5019992414,3cyclictest0-21swapper/319:25:303
179112240,1sleep29-21rcu_preempt23:15:272
151992241,19sleep215300-21/usr/sbin/munin23:10:182
501999231,11cyclictest2154-21kworker/u8:021:37:543
500999233,6cyclictest0-21swapper/122:50:301
500999232,11cyclictest14804-21munin-run00:25:011
500999231,4cyclictest0-21swapper/100:35:151
500999231,11cyclictest140162expr21:55:141
500999231,11cyclictest0-21swapper/122:35:591
500999231,11cyclictest0-21swapper/122:20:071
500999231,10cyclictest68342expr20:25:111
500299230,9cyclictest5240-21kworker/0:122:57:320
501999226,5cyclictest12822-21switchtime21:50:263
501999221,11cyclictest830-21diskstats21:25:143
501399221,5cyclictest26474-21kworker/u8:123:40:212
5013992210,3cyclictest1176-21switchtime20:10:312
501399220,7cyclictest0-21swapper/223:34:202
500999228,3cyclictest0-21swapper/121:13:531
500999221,9cyclictest0-21swapper/100:04:141
500999221,3cyclictest21314-21sh23:25:131
500999221,3cyclictest21314-21sh23:25:131
5009992212,8cyclictest0-21swapper/122:10:221
500999221,17cyclictest0-21swapper/120:09:101
500999221,16cyclictest0-21swapper/121:15:071
500999221,13cyclictest0-21swapper/122:45:141
500999221,11cyclictest0-21swapper/123:41:031
500999221,11cyclictest0-21swapper/122:33:051
500999221,11cyclictest0-21swapper/122:17:301
500299221,9cyclictest24934-21kworker/0:223:32:490
500299221,5cyclictest0-21swapper/021:11:130
500299221,4cyclictest0-21swapper/021:35:130
500299221,12cyclictest15031-21/usr/sbin/munin00:25:250
113242220,4sleep011329-21fschecks_count20:35:160
106942220,2sleep00-21swapper/023:00:160
501999218,4cyclictest0-21swapper/323:10:263
501999218,3cyclictest17407-21cat20:50:023
501999217,3cyclictest8124-21df_inode22:55:143
501999211,11cyclictest14679-21/usr/sbin/munin19:30:243
501999211,11cyclictest0-21swapper/322:15:263
501999210,11cyclictest0-21swapper/322:25:443
500999215,3cyclictest26065-21diskmemload21:40:491
500999211,5cyclictest0-21swapper/123:38:351
5009992115,3cyclictest20857-21proc_pri20:55:211
500999211,3cyclictest1697-21sed23:55:021
500999211,2cyclictest0-21swapper/119:10:151
500999211,18cyclictest0-21swapper/120:19:091
500999211,18cyclictest0-21swapper/120:03:191
500999211,18cyclictest0-21swapper/119:54:321
500999211,15cyclictest2688-21kworker/1:221:47:161
500999211,15cyclictest0-21swapper/121:29:171
500999211,15cyclictest0-21swapper/119:28:051
500999211,13cyclictest0-21swapper/100:28:201
500999211,11cyclictest0-21swapper/121:32:381
500999211,10cyclictest0-21swapper/123:49:191
500999210,3cyclictest0-21swapper/123:57:241
500999210,15cyclictest0-21swapper/119:30:151
500299218,8cyclictest17177-21sh22:00:240
45112210,4sleep011-21rcuc/022:45:230
322622211,2sleep10-21swapper/120:10:061
501999204,7cyclictest41-21ksoftirqd/320:05:283
501999204,6cyclictest26474-21kworker/u8:122:30:133
501999203,8cyclictest25065-21cat22:20:143
501999201,13cyclictest0-21swapper/322:10:243
501999201,11cyclictest9441-21netstat00:10:193
501999201,11cyclictest23172-21/usr/sbin/munin23:30:193
501999201,11cyclictest0-21swapper/321:30:173
501999201,11cyclictest0-21swapper/300:22:053
501999201,10cyclictest0-21swapper/300:19:183
501999200,9cyclictest0-21swapper/323:25:023
501999200,9cyclictest0-21swapper/323:25:013
501999200,9cyclictest0-21swapper/300:37:503
501999200,9cyclictest0-21swapper/300:05:533
501999200,9cyclictest0-21swapper/300:02:443
500999206,8cyclictest0-21swapper/122:44:511
5009992016,3cyclictest0-21swapper/122:08:211
500999201,17cyclictest13474-21seq21:55:001
500999201,17cyclictest0-21swapper/120:43:201
500999201,17cyclictest0-21swapper/120:36:431
500999201,11cyclictest17569-21df_inode00:30:161
500999201,11cyclictest16588-21/usr/sbin/munin23:15:241
500999201,11cyclictest15645-21sensors_temp23:10:231
500999201,10cyclictest8107-21grep19:15:121
500999201,10cyclictest0-21swapper/122:57:441
500299208,2cyclictest19164-21kworker/0:100:36:100
500299207,2cyclictest8190-21kworker/0:200:10:260
500299202,9cyclictest20969-21kworker/0:119:54:080
500299201,11cyclictest424-21/usr/sbin/munin21:25:170
500299201,11cyclictest2725-21/usr/sbin/munin21:30:250
500299201,11cyclictest24755-21/usr/sbin/munin22:20:200
120282201,3sleep10-21swapper/119:22:151
501999198,3cyclictest25370-21/usr/sbin/munin23:35:143
501999198,3cyclictest13714-21/usr/sbin/munin21:55:123
501999197,4cyclictest0-21swapper/323:50:153
501999197,3cyclictest4552-21grep22:45:243
501999196,4cyclictest0-21swapper/323:21:223
501999195,5cyclictest0-21swapper/322:39:563
501999194,2cyclictest87350irq/126-enp1s0-22:41:423
501999193,8cyclictest41-21ksoftirqd/322:05:253
501999192,9cyclictest41-21ksoftirqd/322:51:053
501999192,5cyclictest0-21swapper/321:49:493
501999191,3cyclictest18504-21runrttasks23:19:323
501999191,12cyclictest0-21swapper/320:40:213
501999191,10cyclictest19692-21/usr/sbin/munin20:55:223
501999191,10cyclictest0-21swapper/323:05:413
501999190,4cyclictest41-21ksoftirqd/319:45:013
501399191,8cyclictest29747-21grep22:30:162
501399191,4cyclictest4427-21kworker/u8:221:35:062
501399191,4cyclictest29593-21pluginstate21:15:222
501399191,3cyclictest4313-21users21:30:272
501399191,10cyclictest0-21swapper/200:37:562
500999198,3cyclictest17091-21expr19:35:121
500999191,8cyclictest0-21swapper/121:38:041
5009991916,2cyclictest0-21swapper/121:03:141
5009991916,2cyclictest0-21swapper/121:03:141
5009991915,3cyclictest0-21swapper/100:15:501
500999191,16cyclictest0-21swapper/123:01:031
500999191,10cyclictest396-21cat21:25:021
500999191,10cyclictest27413-21sshd19:57:171
500999191,10cyclictest12535-21sh23:05:141
500999191,10cyclictest0-21swapper/123:20:061
500999190,18cyclictest0-21swapper/100:09:211
500299198,3cyclictest12831-21/usr/sbin/munin00:20:250
500299197,3cyclictest21887-21/usr/sbin/munin21:00:230
500299197,3cyclictest21887-21/usr/sbin/munin21:00:230
500299192,8cyclictest4003-21/usr/sbin/munin00:00:180
500299191,10cyclictest28490-21/usr/sbin/munin21:15:140
500299191,10cyclictest10637-21/usr/sbin/munin00:15:140
500299190,7cyclictest0-21swapper/020:15:150
62222180,2sleep00-21swapper/000:05:060
501999188,3cyclictest41-21ksoftirqd/323:00:143
501999188,2cyclictest30760-21strings23:45:193
501999183,9cyclictest1455650kworker/u8:200:25:283
5019991814,3cyclictest0-21swapper/321:06:253
5019991813,3cyclictest5304-21cut20:20:193
501999181,15cyclictest0-21swapper/319:56:023
501999180,10cyclictest0-21swapper/321:10:133
501399181,8cyclictest0-21swapper/223:55:242
501399181,7cyclictest0-21swapper/221:52:152
501399181,7cyclictest0-21swapper/221:12:132
501399181,6cyclictest0-21swapper/222:45:282
501399181,6cyclictest0-21swapper/200:19:052
501399181,5cyclictest0-21swapper/200:11:482
501399181,3cyclictest87350irq/126-enp1s0-22:19:072
501399181,3cyclictest0-21swapper/223:48:572
501399181,3cyclictest0-21swapper/222:11:582
5013991813,4cyclictest0-21swapper/222:05:132
500999183,6cyclictest0-21swapper/123:30:551
500999182,13cyclictest4442-21cpuspeed_turbos20:20:111
5009991815,2cyclictest0-21swapper/119:45:431
5009991814,3cyclictest0-21swapper/120:31:201
500999181,16cyclictest0-21swapper/120:45:541
500999181,16cyclictest0-21swapper/119:40:401
500299188,2cyclictest20978-21/usr/sbin/munin23:25:190
500299188,2cyclictest20978-21/usr/sbin/munin23:25:190
500299187,2cyclictest31488-21kworker/0:123:49:390
500299182,4cyclictest2195-21cut23:55:160
500299181,9cyclictest808-21snmpd21:05:040
500299181,7cyclictest0-21swapper/022:43:230
500299181,6cyclictest0-21swapper/022:52:580
5002991813,3cyclictest808-21snmpd22:05:150
500299181,15cyclictest10295-21users20:30:270
500299180,8cyclictest0-21swapper/023:17:400
193652180,4sleep230-21rcuc/219:40:132
501999177,3cyclictest87350irq/126-enp1s0-19:20:203
501999173,11cyclictest4102-21grep20:20:023
501999172,3cyclictest23995-21cat21:05:023
501999172,3cyclictest23995-21cat21:05:013
501999172,2cyclictest39-21rcuc/323:40:243
501999171,9cyclictest0-21swapper/300:31:313
501999171,7cyclictest0-21swapper/323:55:163
501999171,4cyclictest113282sort20:35:173
501999171,3cyclictest8958-21chrt21:43:263
501999171,3cyclictest29486-21sshd20:00:413
5019991712,3cyclictest0-21swapper/319:36:493
501999171,14cyclictest0-21swapper/320:50:103
501999171,11cyclictest0-21swapper/321:20:463
501999171,10cyclictest0-21swapper/321:15:233
501999171,10cyclictest0-21swapper/320:32:063
501999171,10cyclictest0-21swapper/319:15:073
501399176,3cyclictest87650irq/129-enp1s0-00:30:202
501399173,3cyclictest1658-21pmu-power21:25:232
501399171,8cyclictest0-21swapper/222:29:402
501399171,7cyclictest0-21swapper/223:51:232
501399171,6cyclictest22511-21expr21:00:152
501399171,6cyclictest22511-21expr21:00:152
501399171,6cyclictest0-21swapper/222:35:132
501399171,3cyclictest14679-21/usr/sbin/munin19:30:232
501399171,2cyclictest4615-21cat20:20:122
5013991712,3cyclictest16877-21/usr/sbin/munin19:35:212
5013991711,4cyclictest909-21sssd_nss21:05:222
501399171,14cyclictest0-21swapper/221:55:102
501399171,14cyclictest0-21swapper/220:16:302
5013991711,3cyclictest9979-21/usr/sbin/munin23:00:202
5013991711,3cyclictest25588-21expr23:35:112
5013991711,3cyclictest24115-21grep19:50:162
5013991711,3cyclictest12032-21pmu-power20:35:242
5013991710,4cyclictest909-21sssd_nss00:00:152
5013991710,4cyclictest2517-21cat22:40:272
500299177,7cyclictest0-21swapper/019:30:250
500299171,8cyclictest0-21swapper/023:42:080
500299171,6cyclictest0-21swapper/023:51:110
500299171,3cyclictest808-21snmpd22:35:210
500299171,15cyclictest0-21swapper/020:23:030
500299171,13cyclictest30076-21/usr/sbin/munin20:05:190
5002991711,3cyclictest19410-21grep23:20:180
500299171,10cyclictest0-21swapper/023:09:120
500299170,15cyclictest0-21swapper/019:20:550
500299170,11cyclictest0-21swapper/021:40:100
159692170,2sleep20-21swapper/222:00:122
124692170,2sleep00-21swapper/019:25:020
501999163,2cyclictest23851-21fschecks_count19:50:153
5019991611,3cyclictest7313-21munin-node20:25:113
5019991611,3cyclictest32269-21/usr/sbin/munin20:10:173
501399169,4cyclictest20978-21/usr/sbin/munin23:25:112
501399169,4cyclictest20978-21/usr/sbin/munin23:25:112
501399165,3cyclictest87350irq/126-enp1s0-19:20:172
501399161,7cyclictest0-21swapper/200:25:092
501399161,6cyclictest30419-21/usr/sbin/munin20:05:142
501399161,6cyclictest16855-21sed20:45:262
501399161,6cyclictest0-21swapper/200:22:012
501399161,5cyclictest5526-21cat19:10:122
501399161,5cyclictest0-21swapper/220:50:092
5013991612,3cyclictest0-21swapper/219:59:392
5013991612,3cyclictest0-21swapper/219:17:122
501399161,14cyclictest0-21swapper/221:40:312
501399161,14cyclictest0-21swapper/220:57:542
501399161,14cyclictest0-21swapper/219:26:582
5013991611,3cyclictest6508-21/usr/sbin/munin20:25:242
5013991611,3cyclictest14075-21/usr/sbin/munin20:40:172
5013991610,3cyclictest25106-21entropy22:20:152
5013991610,3cyclictest10805-21unixbench_multi21:45:272
501399160,6cyclictest0-21swapper/219:45:202
501399160,14cyclictest0-21swapper/220:30:062
500299165,3cyclictest165050irq/12-ttyS020:00:280
500299164,4cyclictest27304-21diskstats22:25:150
500299163,3cyclictest87350irq/126-enp1s0-21:20:170
500299161,7cyclictest0-21swapper/000:30:150
500299161,5cyclictest21745-21uniq19:45:140
500299161,5cyclictest0-21swapper/023:12:420
5002991612,3cyclictest0-21swapper/021:53:150
5002991612,3cyclictest0-21swapper/020:49:040
5002991611,3cyclictest7932-21sed20:25:250
5002991611,3cyclictest29152-21acpi22:30:120
5002991611,3cyclictest13097-21/usr/sbin/munin20:40:150
500299161,12cyclictest16877-21/usr/sbin/munin19:35:160
5002991610,3cyclictest953-21ntpq20:10:200
5002991610,3cyclictest6497-21cat19:10:230
501999159,3cyclictest5198-21/usr/sbin/munin19:10:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional