You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-16 - 07:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Tue Jun 16, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
127812230162,55sleep20-21swapper/219:09:492
127422225195,20sleep00-21swapper/019:09:190
126512203167,24sleep10-21swapper/119:08:071
125582203167,24sleep30-21swapper/319:06:573
1457921530,2sleep121-21rcuc/120:25:131
1719321410,2sleep31289799cyclictest19:15:233
87212720,1sleep20-21swapper/221:25:132
279702530,9sleep341-21ksoftirqd/322:10:013
149732510,3sleep00-21swapper/020:25:160
1288099348,12cyclictest30784-21munin-run23:30:000
12897993211,9cyclictest0-21swapper/321:34:553
1289799321,12cyclictest0-21swapper/323:08:063
1288099329,11cyclictest0-21swapper/021:15:000
12897993116,3cyclictest0-21swapper/322:21:273
1288099306,11cyclictest10990-21idleruntime-cro22:45:020
12897992918,4cyclictest0-21swapper/300:31:263
12897992918,4cyclictest0-21swapper/300:31:263
12897992915,3cyclictest0-21swapper/321:42:393
1289799291,11cyclictest0-21swapper/319:45:173
1289799291,10cyclictest0-21swapper/300:25:313
12897992817,3cyclictest0-21swapper/321:20:313
12897992813,3cyclictest0-21swapper/321:11:343
1289799281,12cyclictest0-21swapper/323:18:463
12897992810,4cyclictest4303-21smart_sda23:40:233
1289799280,19cyclictest0-21swapper/300:39:503
1289799280,12cyclictest0-21swapper/322:57:223
1288099280,13cyclictest6596-21idleruntime-cro22:35:020
12897992715,3cyclictest0-21swapper/322:03:393
12897992714,3cyclictest0-21swapper/323:36:393
1289799271,11cyclictest0-21swapper/321:59:513
1289799271,11cyclictest0-21swapper/300:05:013
1288099272,12cyclictest31136-21awk19:50:010
12897992615,3cyclictest0-21swapper/322:34:233
1289799261,15cyclictest0-21swapper/322:39:223
1289799261,11cyclictest0-21swapper/320:55:143
1289799261,10cyclictest0-21swapper/322:26:073
1289799261,10cyclictest0-21swapper/300:24:113
1289799260,9cyclictest0-21swapper/320:11:273
1289799260,13cyclictest0-21swapper/322:53:463
1289799260,11cyclictest0-21swapper/323:13:103
12886992616,3cyclictest0-21swapper/122:46:511
1288099265,10cyclictest18782-21/usr/sbin/munin20:35:190
1288099262,11cyclictest15000-21/usr/sbin/munin21:40:120
1288099260,13cyclictest11622-21awk00:00:010
1289799258,10cyclictest0-21swapper/322:49:303
12897992511,3cyclictest0-21swapper/323:45:353
1289799251,11cyclictest0-21swapper/319:55:073
12897992510,11cyclictest0-21swapper/300:19:113
1289799250,9cyclictest0-21swapper/322:41:043
1289399251,6cyclictest0-21swapper/200:14:562
1289399251,6cyclictest0-21swapper/200:04:112
1289399250,10cyclictest0-21swapper/222:08:322
1288699255,6cyclictest26568-21grep22:05:151
1288699252,18cyclictest21306-40dnf21:54:241
12886992511,12cyclictest19680-21kworker/u8:123:25:011
12886992511,12cyclictest19680-21kworker/u8:123:25:011
12880992519,3cyclictest3716-21/usr/sbin/munin23:40:140
12880992511,2cyclictest87350irq/126-enp1s0-23:36:260
1289799247,3cyclictest0-21swapper/320:39:143
1289799247,3cyclictest0-21swapper/320:08:473
12897992415,3cyclictest0-21swapper/323:33:513
12897992414,3cyclictest0-21swapper/321:45:393
12897992413,3cyclictest0-21swapper/319:52:393
12897992413,3cyclictest0-21swapper/319:52:393
1289399244,9cyclictest32-21ksoftirqd/221:50:012
1289399241,6cyclictest0-21swapper/221:50:172
1288699241,17cyclictest0-21swapper/122:52:401
1288699240,17cyclictest0-21swapper/100:33:351
1288699240,17cyclictest0-21swapper/100:33:351
1288099245,8cyclictest22007-21awk23:10:000
1288099241,6cyclictest0-21swapper/023:32:540
1288099241,12cyclictest27975-21munin-run22:10:010
67412230,4sleep11288699cyclictest21:20:151
225722230,2sleep20-21swapper/220:40:272
1289799239,8cyclictest0-21swapper/321:15:463
1289799239,3cyclictest0-21swapper/321:04:543
1289799238,3cyclictest0-21swapper/320:33:343
1289799235,3cyclictest0-21swapper/320:50:503
1289799234,3cyclictest87350irq/126-enp1s0-00:05:133
1289799231,11cyclictest808-21snmpd21:50:153
1289799231,10cyclictest0-21swapper/321:36:313
12897992310,3cyclictest0-21swapper/321:09:223
1289799230,10cyclictest0-21swapper/300:13:113
1289399231,6cyclictest30-21rcuc/223:01:472
1289399231,5cyclictest0-21swapper/221:58:312
1289399231,4cyclictest0-21swapper/221:20:152
1289399231,3cyclictest0-21swapper/222:41:242
1288699238,12cyclictest30039-21latency19:45:171
1288699232,18cyclictest32075-21ntp_states23:30:241
1288699231,3cyclictest827-21runrttasks23:57:381
1288699231,11cyclictest0-21swapper/123:10:271
1288699230,1cyclictest0-21swapper/119:25:101
1288099239,4cyclictest16800-21kworker/u8:022:14:540
1288099239,4cyclictest16800-21kworker/u8:022:14:540
1288099237,3cyclictest2273-21latency_hist22:25:020
1288099233,3cyclictest0-21swapper/022:37:530
1288099231,3cyclictest0-21swapper/022:25:240
1288099231,13cyclictest0-21swapper/000:00:290
1288099231,11cyclictest946-21munin-run19:55:010
1288099231,11cyclictest946-21munin-run19:55:010
1288099230,5cyclictest0-21swapper/000:12:060
1289799227,3cyclictest0-21swapper/323:21:543
1289799227,3cyclictest0-21swapper/323:21:543
1289799224,2cyclictest0-21swapper/323:57:193
12897992211,3cyclictest0-21swapper/322:17:313
12897992211,3cyclictest0-21swapper/320:46:343
1289799221,11cyclictest0-21swapper/319:29:053
12897992210,3cyclictest0-21swapper/320:26:223
1289799220,9cyclictest0-21swapper/319:10:243
1289799220,5cyclictest30601-21kworker/u8:023:50:113
1289399221,7cyclictest0-21swapper/200:32:382
1289399221,7cyclictest0-21swapper/200:32:382
1289399221,6cyclictest0-21swapper/223:15:302
1289399221,6cyclictest0-21swapper/222:36:072
1289399221,6cyclictest0-21swapper/222:22:392
1289399221,6cyclictest0-21swapper/221:35:242
1289399221,6cyclictest0-21swapper/200:27:542
1289399221,5cyclictest9-21rcu_preempt21:10:212
1289399221,5cyclictest19680-21kworker/u8:123:40:522
1289399221,5cyclictest0-21swapper/223:07:052
1289399221,5cyclictest0-21swapper/222:13:482
1289399221,5cyclictest0-21swapper/222:13:482
1289399221,4cyclictest9164-21kworker/u8:223:57:232
1288699229,6cyclictest21064-21kworker/u8:120:14:291
1288699227,13cyclictest11632-21kworker/u8:300:22:561
1288699224,10cyclictest1284-21ntp_states22:20:211
1288699223,17cyclictest23-21ksoftirqd/121:34:091
12886992212,3cyclictest5277-21smart_sda21:15:251
1288699221,14cyclictest23563-21latency_hist22:00:001
1288699221,13cyclictest0-21swapper/121:42:581
1288699220,17cyclictest0-21swapper/122:17:461
1288099226,3cyclictest24141-21diskstats22:00:120
1288099221,8cyclictest5082-21ntp_states21:15:220
1288099221,5cyclictest0-21swapper/021:58:580
1288099221,2cyclictest87350irq/126-enp1s0-23:04:490
12880992211,5cyclictest87350irq/126-enp1s0-23:18:390
1288099221,10cyclictest30396-21/usr/sbin/munin22:15:260
94002211,3sleep09422-21awk23:55:010
256622210,3sleep30-21swapper/319:35:193
178812211,2sleep10-21swapper/119:19:241
1289799216,3cyclictest0-21swapper/320:43:583
1289799216,3cyclictest0-21swapper/319:44:013
1289799216,3cyclictest0-21swapper/319:30:353
12897992113,3cyclictest0-21swapper/322:14:433
12897992113,3cyclictest0-21swapper/322:14:433
1289799211,13cyclictest0-21swapper/323:01:303
1289799211,10cyclictest0-21swapper/321:27:433
1289399213,6cyclictest89250irq/135-enp2s0-20:20:122
1289399211,6cyclictest0-21swapper/223:48:042
1289399211,6cyclictest0-21swapper/222:57:562
1289399211,5cyclictest0-21swapper/223:10:022
1289399211,5cyclictest0-21swapper/222:54:522
1289399211,4cyclictest0-21swapper/222:25:032
1289399211,3cyclictest27693-21kworker/u8:122:33:012
12893992111,3cyclictest8616-21processes23:50:222
1288699218,8cyclictest0-21swapper/121:39:491
1288699215,2cyclictest9417-21perf23:55:011
1288699213,16cyclictest0-21swapper/100:35:181
1288699212,5cyclictest1589-21perf21:10:011
1288699211,9cyclictest0-21swapper/122:59:511
1288699211,4cyclictest3448-21taskset21:10:491
1288699211,17cyclictest26439-21ntp_states20:50:211
1288699211,15cyclictest0-21swapper/100:00:231
12886992110,3cyclictest18229-21latency_hist00:15:001
1288699210,9cyclictest21-21rcuc/100:27:271
1288699210,12cyclictest0-21swapper/122:40:481
1288099217,3cyclictest31973-21/usr/sbin/munin21:05:180
1288099213,3cyclictest0-21swapper/022:45:070
1288099213,15cyclictest12577-21munin-run21:35:010
12880992111,7cyclictest21306-40dnf21:54:240
1288099211,17cyclictest16692-21ntp_states22:55:230
1288099211,11cyclictest0-21swapper/000:32:110
1288099211,11cyclictest0-21swapper/000:32:110
1289799207,3cyclictest0-21swapper/320:19:423
1289799207,3cyclictest0-21swapper/319:20:473
1289799202,10cyclictest28814-21/usr/sbin/munin23:25:213
1289399201,6cyclictest26845-21grep20:50:252
1289399201,6cyclictest0-21swapper/219:20:192
1289399201,5cyclictest0-21swapper/223:39:222
1289399201,5cyclictest0-21swapper/223:30:452
1289399201,5cyclictest0-21swapper/223:29:042
1289399201,5cyclictest0-21swapper/222:16:132
1289399201,5cyclictest0-21swapper/222:03:482
1289399201,5cyclictest0-21swapper/200:23:562
1289399201,5cyclictest0-21swapper/200:16:112
1289399201,4cyclictest0-21swapper/221:17:472
1289399201,4cyclictest0-21swapper/200:37:592
12893992013,4cyclictest15886-21grep20:25:252
1288699205,7cyclictest0-21swapper/122:26:381
1288699205,6cyclictest28199-21/usr/sbin/munin22:10:211
1288699205,6cyclictest28199-21/usr/sbin/munin22:10:211
1288699203,3cyclictest221ktimersoftd/123:25:131
1288699202,6cyclictest755-21rngd23:15:471
12886992016,2cyclictest6765-21ps20:05:241
1288699201,18cyclictest0-21swapper/119:34:421
1288699201,12cyclictest0-21swapper/122:37:081
1288699201,11cyclictest0-21swapper/123:07:011
1288699201,10cyclictest0-21swapper/120:55:171
1288099206,7cyclictest0-21swapper/021:46:330
1288099203,6cyclictest0-21swapper/021:38:560
1288099203,14cyclictest26974-21/usr/sbin/munin23:20:130
1288099203,14cyclictest26974-21/usr/sbin/munin23:20:130
1288099202,5cyclictest22850-21/usr/sbin/munin00:25:150
1288099201,6cyclictest22223-21/usr/sbin/munin23:10:200
1288099201,5cyclictest27249-21/usr/sbin/munin00:35:130
1288099201,4cyclictest0-21swapper/000:20:330
1288099201,3cyclictest0-21swapper/000:08:280
12880992012,7cyclictest21064-21kworker/u8:120:03:300
1289799196,3cyclictest0-21swapper/320:20:063
1289799195,3cyclictest0-21swapper/320:02:533
1289399193,5cyclictest0-21swapper/222:48:542
1289399191,6cyclictest0-21swapper/219:16:212
1289399191,5cyclictest18245-1kworker/u9:223:24:592
1289399191,5cyclictest18245-1kworker/u9:223:24:592
1289399191,10cyclictest0-21swapper/220:04:432
1288699199,3cyclictest0-21swapper/122:02:521
1288699191,5cyclictest19132-21hddtemp_smartct00:15:151
12886991915,3cyclictest0-21swapper/119:35:201
1288699191,3cyclictest0-21swapper/119:52:371
1288699191,3cyclictest0-21swapper/119:52:371
1288699191,17cyclictest0-21swapper/119:12:241
1288699191,16cyclictest0-21swapper/121:45:191
1288699191,16cyclictest0-21swapper/120:20:081
1288699191,16cyclictest0-21swapper/100:06:031
1288699191,12cyclictest0-21swapper/123:00:101
1288699190,17cyclictest0-21swapper/120:40:231
1288699190,14cyclictest0-21swapper/123:45:291
1288099198,8cyclictest27207-40dnf20:53:250
1288099193,7cyclictest21291-21ls20:40:130
1289399181,7cyclictest16926-21df_inode20:30:132
1289399181,6cyclictest0-21swapper/221:31:322
12893991811,4cyclictest0-21swapper/219:10:152
1288699184,3cyclictest0-21swapper/121:00:291
12886991814,3cyclictest0-21swapper/120:36:041
12886991814,3cyclictest0-21swapper/119:56:331
1288699181,3cyclictest0-21swapper/119:23:301
12886991812,4cyclictest10144-21tr20:15:131
12886991812,3cyclictest1173-21diskstats23:35:141
1288699181,16cyclictest0-21swapper/121:26:051
1288699181,16cyclictest0-21swapper/120:47:531
1288699181,16cyclictest0-21swapper/120:30:181
1288699181,16cyclictest0-21swapper/120:00:471
1288699181,16cyclictest0-21swapper/119:41:391
1288699180,4cyclictest0-21swapper/123:41:221
1288099189,6cyclictest11997-21cut20:20:020
1288099189,4cyclictest14165-21munin-run20:25:020
1288099185,3cyclictest0-21swapper/021:23:020
1288099181,5cyclictest26955-21/usr/sbin/munin19:40:130
1288099181,5cyclictest0-21swapper/020:10:140
1288099181,4cyclictest5599-21/usr/sbin/munin20:05:270
12880991812,4cyclictest17944-21ls19:20:020
1288099181,10cyclictest0-21swapper/019:20:200
1288099180,6cyclictest11-21rcuc/020:55:150
1288099180,5cyclictest24764-21/usr/sbin/munin19:35:160
117202180,2sleep20-21swapper/220:17:072
1289399171,5cyclictest0-21swapper/221:40:122
1289399171,15cyclictest0-21swapper/219:58:242
1289399171,14cyclictest0-21swapper/220:55:032
1289399171,14cyclictest0-21swapper/219:35:562
12893991711,3cyclictest31349-21/usr/sbin/munin19:50:182
12893991711,3cyclictest31349-21/usr/sbin/munin19:50:182
1289399171,10cyclictest0-21swapper/220:36:482
12893991710,4cyclictest0-21swapper/200:05:212
1288699171,2cyclictest0-21swapper/122:30:461
1288099177,4cyclictest87650irq/129-enp1s0-20:30:240
1288099173,11cyclictest23172-21/usr/sbin/munin20:45:130
1288099172,12cyclictest8406-21/usr/sbin/munin21:25:190
1288099171,6cyclictest0-21swapper/023:45:070
1288099171,5cyclictest0-21swapper/022:50:300
1288099171,15cyclictest0-21swapper/019:27:130
1288099171,13cyclictest808-21snmpd21:00:180
12880991711,3cyclictest1545-21fschecks_count19:55:150
12880991710,4cyclictest23475-21load19:30:170
1288099170,6cyclictest0-21swapper/019:10:250
1288099170,5cyclictest0-21swapper/000:16:350
1289399164,6cyclictest790-21in:imjournal19:45:112
1289399161,9cyclictest0-21swapper/220:10:272
1289399161,6cyclictest27764-21irqstats19:40:152
1289399161,6cyclictest0-21swapper/220:05:132
1289399161,5cyclictest31288-21grep21:00:252
1289399161,3cyclictest32-21ksoftirqd/219:27:102
12893991611,3cyclictest23433-21ls20:45:122
12893991610,4cyclictest0-21swapper/221:05:152
1289399160,4cyclictest0-21swapper/219:30:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional