You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-09 - 16:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat May 09, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2070999260,25cyclictest8850-21rm09:33:210
2070999260,25cyclictest14006-21diskmemload12:32:210
20723992523,1cyclictest0-21swapper/312:18:223
2072399251,23cyclictest23012-21kworker/u8:212:39:363
2072399250,24cyclictest9055-21ssh12:24:503
20719992521,3cyclictest0-21swapper/209:50:122
2071999250,24cyclictest14006-21diskmemload09:16:412
20712992523,1cyclictest0-21swapper/111:11:291
2071299250,24cyclictest2204-21sh12:18:221
2071299250,24cyclictest0-21swapper/111:48:371
2071299250,1cyclictest0-21swapper/112:24:501
2070999253,21cyclictest62550irq/127-eno110:50:580
2072399240,23cyclictest19321-21ssh12:05:183
2072399240,1cyclictest20511-21rm11:37:143
20719992422,1cyclictest21411-21ssh11:38:312
20719992419,2cyclictest0-21swapper/209:26:322
2071999240,23cyclictest14006-21diskmemload11:31:342
2071999240,23cyclictest0-21swapper/212:29:102
2071999240,23cyclictest0-21swapper/212:12:382
2071999240,23cyclictest0-21swapper/211:13:102
2071999240,23cyclictest0-21swapper/210:35:592
2071999240,1cyclictest22866-21rm10:14:042
2071999240,1cyclictest0-21swapper/210:29:592
20712992423,1cyclictest20309-21apt-get12:06:411
20712992421,2cyclictest28050-21ssh10:18:221
2071299240,2cyclictest2000-21sh09:56:231
2071299240,24cyclictest0-21swapper/112:01:071
2071299240,23cyclictest4373-21rm10:26:421
2071299240,23cyclictest0-21swapper/110:38:241
2071299240,1cyclictest10676-21kworker/u8:009:42:191
2071299240,0cyclictest14006-21diskmemload10:53:091
20709992423,1cyclictest62550irq/127-eno112:01:500
20709992423,1cyclictest62550irq/127-eno110:45:370
2070999240,24cyclictest14006-21diskmemload11:43:160
2070999240,24cyclictest0-21swapper/011:08:490
2070999240,24cyclictest0-21swapper/010:23:300
2070999240,24cyclictest0-21swapper/009:40:390
2070999240,23cyclictest8647-21ssh10:30:270
2070999240,23cyclictest0-21swapper/009:13:120
2070999240,0cyclictest14006-21diskmemload12:05:230
2070999240,0cyclictest0-21swapper/012:27:420
2070999240,0cyclictest0-21swapper/010:28:090
20723992323,0cyclictest20828-21apt-get10:41:403
20723992323,0cyclictest0-21swapper/312:01:073
20723992323,0cyclictest0-21swapper/309:33:393
20723992321,1cyclictest15849-21ssh11:33:413
2072399230,2cyclictest27055-21ssh09:49:593
2072399230,2cyclictest2515-21rm10:53:053
2072399230,23cyclictest0-21swapper/311:20:513
2072399230,23cyclictest0-21swapper/311:07:343
2072399230,23cyclictest0-21swapper/310:37:153
2072399230,22cyclictest5537-21ssh09:58:583
2072399230,22cyclictest25704-21ssh10:45:393
2072399230,22cyclictest22010-21ssh10:12:523
2072399230,22cyclictest14651-21ssh09:10:293
2072399230,22cyclictest14006-21diskmemload11:11:023
2072399230,1cyclictest25168-21ssh12:10:333
2072399230,1cyclictest21363-21apt-get09:16:423
2072399230,0cyclictest14006-21diskmemload10:30:383
2072399230,0cyclictest0-21swapper/311:48:113
2072399230,0cyclictest0-21swapper/310:25:563
20719992323,0cyclictest0-21swapper/211:05:122
20719992323,0cyclictest0-21swapper/210:40:422
20719992323,0cyclictest0-21swapper/210:19:182
20719992323,0cyclictest0-21swapper/209:24:422
20719992322,1cyclictest0-21swapper/211:25:012
20719992321,1cyclictest14006-21diskmemload12:33:002
20719992321,1cyclictest0-21swapper/212:01:462
20719992321,1cyclictest0-21swapper/211:00:542
20719992321,1cyclictest0-21swapper/210:33:172
20719992321,1cyclictest0-21swapper/209:10:452
20719992320,2cyclictest28918-21ssh11:45:412
20719992319,4cyclictest0-21swapper/210:46:092
2071999231,21cyclictest0-21swapper/212:06:122
2071999230,23cyclictest0-21swapper/212:35:432
2071999230,23cyclictest0-21swapper/211:22:422
2071999230,22cyclictest8853-21awk09:00:002
2071999230,22cyclictest12146-21wc10:05:012
2071999230,22cyclictest0-21swapper/211:19:442
2071999230,22cyclictest0-21swapper/209:41:002
2071999230,1cyclictest30266-21idleruntime-cro07:30:012
2071999230,1cyclictest0-21swapper/209:34:422
2071999230,0cyclictest0-21swapper/211:56:422
2071999230,0cyclictest0-21swapper/211:54:122
2071999230,0cyclictest0-21swapper/209:57:402
20712992323,0cyclictest0-21swapper/111:54:071
20712992323,0cyclictest0-21swapper/111:29:311
20712992323,0cyclictest0-21swapper/109:22:501
20712992321,1cyclictest0-21swapper/110:31:131
20712992321,1cyclictest0-21swapper/109:12:211
2071299230,23cyclictest14006-21diskmemload12:37:481
2071299230,23cyclictest0-21swapper/111:04:251
2071299230,23cyclictest0-21swapper/110:20:161
2071299230,23cyclictest0-21swapper/110:13:261
2071299230,23cyclictest0-21swapper/110:00:431
2071299230,22cyclictest0-21swapper/111:40:211
2071299230,22cyclictest0-21swapper/111:35:061
2071299230,22cyclictest0-21swapper/111:30:271
2071299230,22cyclictest0-21swapper/109:30:031
2071299230,0cyclictest0-21swapper/112:32:011
2071299230,0cyclictest0-21swapper/112:25:301
2071299230,0cyclictest0-21swapper/110:46:421
2071299230,0cyclictest0-21swapper/110:44:151
2071299230,0cyclictest0-21swapper/109:49:471
20709992323,0cyclictest14006-21diskmemload11:36:290
20709992323,0cyclictest14006-21diskmemload10:19:160
20709992323,0cyclictest0-21swapper/011:57:220
20709992323,0cyclictest0-21swapper/011:33:550
20709992323,0cyclictest0-21swapper/011:19:190
20709992322,1cyclictest21298-21ssh12:35:460
20709992321,2cyclictest17509-21ssh10:37:260
20709992321,1cyclictest25009-21ssh11:12:410
20709992321,1cyclictest14006-21diskmemload09:37:000
20709992320,2cyclictest16951-21rm10:08:510
2070999230,23cyclictest0-21swapper/009:20:530
2070999230,22cyclictest28504-21ssh12:12:410
2070999230,22cyclictest23598-21ssh09:17:290
2070999230,22cyclictest22107-21sh09:45:330
2070999230,22cyclictest14006-21diskmemload12:20:380
2070999230,22cyclictest14006-21diskmemload11:25:030
2070999230,1cyclictest27678-21taskset09:50:260
2070999230,1cyclictest0-21swapper/009:59:210
2070999230,0cyclictest0-21swapper/011:46:180
20723992222,0cyclictest0-21swapper/311:02:273
20723992221,0cyclictest0-21swapper/308:51:403
20723992220,2cyclictest27779-21ssh11:44:253
20723992220,1cyclictest14424-21ssh12:29:303
20723992220,1cyclictest12738-21ssh10:05:283
2072399220,22cyclictest0-21swapper/309:25:213
2072399220,21cyclictest7012-21ssh10:00:333
2072399220,21cyclictest1862-21ssh11:50:193
2072399220,21cyclictest16229-21ssh12:31:333
2072399220,21cyclictest14006-21diskmemload11:55:013
2072399220,21cyclictest14006-21diskmemload11:28:593
2072399220,21cyclictest14006-21diskmemload10:55:383
2072399220,21cyclictest12766-21apt-get08:01:433
2072399220,21cyclictest10483-21rm09:35:223
2072399220,21cyclictest0-21swapper/311:16:523
2072399220,0cyclictest14006-21diskmemload10:15:113
2072399220,0cyclictest14006-21diskmemload09:41:013
2072399220,0cyclictest0-21swapper/310:20:063
20719992222,0cyclictest0-21swapper/210:51:172
20719992222,0cyclictest0-21swapper/209:46:252
20719992220,1cyclictest0-21swapper/210:06:552
20719992220,1cyclictest0-21swapper/209:35:162
2071999220,22cyclictest0-21swapper/211:44:002
2071999220,21cyclictest4179-21sh12:20:392
2071999220,21cyclictest17218-21awk08:10:002
2071999220,21cyclictest14006-21diskmemload10:20:212
2071999220,21cyclictest0-21swapper/208:11:422
2071999220,1cyclictest23057-21turbostat.cron07:15:002
2071999220,0cyclictest0-21swapper/210:55:212
20712992222,0cyclictest0-21swapper/110:57:201
20712992222,0cyclictest0-21swapper/110:06:221
20712992220,1cyclictest8829-21latency_hist09:00:001
20712992220,1cyclictest5169-21apt07:45:011
20712992220,1cyclictest0-21swapper/108:25:011
2071299220,22cyclictest0-21swapper/111:17:401
2071299220,21cyclictest25865-21apt-get07:21:411
2071299220,21cyclictest0-21swapper/112:10:291
2071299220,21cyclictest0-21swapper/109:50:101
2071299220,21cyclictest0-21swapper/109:25:201
2071299220,21cyclictest0-21swapper/109:16:011
2071299220,21cyclictest0-21swapper/108:46:411
2071299220,1cyclictest0-21swapper/111:05:311
20709992222,0cyclictest0-21swapper/010:57:570
20709992221,1cyclictest31028-21ssh12:15:480
20709992221,1cyclictest0-21swapper/009:26:200
20709992220,1cyclictest0-21swapper/010:12:130
2070999220,22cyclictest0-21swapper/011:02:230
2070999220,21cyclictest818-21ssh11:20:580
2070999220,21cyclictest23038-21cron07:15:000
2070999220,1cyclictest0-21swapper/010:40:110
20723992121,0cyclictest0-21swapper/309:50:023
20723992121,0cyclictest0-21swapper/309:21:423
20723992120,1cyclictest21053-21apt-get07:11:413
20723992120,0cyclictest22430-21apt-get08:21:433
20723992118,2cyclictest8830-21latency_hist09:00:003
2072399210,20cyclictest0-21swapper/308:06:413
2072399210,0cyclictest0-21swapper/307:31:413
2072399210,0cyclictest0-21swapper/307:31:413
20719992119,1cyclictest29220-21latency_hist08:35:012
2071999210,0cyclictest0-21swapper/207:31:412
2071999210,0cyclictest0-21swapper/207:31:412
20712992121,0cyclictest0-21swapper/107:46:411
20712992120,1cyclictest0-21swapper/107:26:421
20712992119,1cyclictest26849-21idleruntime-cro08:30:011
2071299210,20cyclictest0-21swapper/107:11:411
2071299210,0cyclictest0-21swapper/108:51:401
2071299210,0cyclictest0-21swapper/107:16:401
20709992121,0cyclictest0-21swapper/010:00:050
20709992120,1cyclictest1837-21ssh11:50:150
20709992119,1cyclictest32029-21apt-get08:41:410
20709992119,1cyclictest12362-21apt08:00:000
2070999210,0cyclictest0-21swapper/007:41:410
2070999210,0cyclictest0-21swapper/007:36:410
20723992020,0cyclictest0-21swapper/308:46:413
20723992020,0cyclictest0-21swapper/308:16:413
20723992020,0cyclictest0-21swapper/308:11:423
20723992020,0cyclictest0-21swapper/307:21:413
20723992019,1cyclictest8825-21apt-get09:00:023
20723992019,1cyclictest0-21swapper/307:46:413
20723992019,1cyclictest0-21swapper/307:16:403
20723992018,2cyclictest5170-21latency_hist07:45:013
20719992020,0cyclictest0-21swapper/208:01:432
20719992019,1cyclictest0-21swapper/207:16:402
20719992018,1cyclictest12365-21sendmail08:00:002
20712992019,1cyclictest10347-21apt-get07:56:411
2071299200,0cyclictest0-21swapper/108:36:411
2071299200,0cyclictest0-21swapper/107:31:411
2071299200,0cyclictest0-21swapper/107:31:411
20709992020,0cyclictest30652-21apt-get07:31:400
20709992020,0cyclictest30652-21apt-get07:31:400
20709992020,0cyclictest1979-21/usr/sbin/munin08:46:410
2070999200,0cyclictest23444-21apt-get07:16:400
2070999200,0cyclictest0-21swapper/009:00:020
20723991919,0cyclictest0-21swapper/307:56:413
20723991919,0cyclictest0-21swapper/307:26:423
20723991917,1cyclictest26843-21munin-run08:30:013
20719991918,0cyclictest7950-21apt-get07:51:422
20719991918,0cyclictest0-21swapper/208:46:412
2071999190,18cyclictest622-21gdbus08:55:002
20712991918,0cyclictest0-21swapper/109:00:021
20712991917,1cyclictest0-21swapper/107:40:011
20719991818,0cyclictest0-21swapper/208:41:412
20719991816,1cyclictest0-21swapper/207:46:422
20712991818,0cyclictest0-21swapper/108:01:431
20709991818,0cyclictest0-21swapper/007:21:410
20723991715,1cyclictest0-21swapper/307:40:013
2072399170,16cyclictest14006-21diskmemload09:10:013
20719991717,0cyclictest20001-21apt-get08:16:412
2071999170,1cyclictest0-21swapper/207:41:412
20712991717,0cyclictest0-21swapper/109:06:421
20709991716,0cyclictest0-21swapper/008:21:430
20709991715,1cyclictest17226-21perf08:10:000
20709991714,2cyclictest13949-21taskset09:09:250
20719991615,1cyclictest0-21swapper/209:01:442
20712991614,1cyclictest29222-21cron08:35:011
20709991616,0cyclictest0-21swapper/008:31:430
20723991513,1cyclictest29208-21idleruntime-cro08:35:013
20719991515,0cyclictest0-21swapper/209:06:422
20719991515,0cyclictest0-21swapper/208:26:412
20719991515,0cyclictest0-21swapper/207:21:422
20719991513,1cyclictest31655-21sendmail-msp08:40:012
20712991515,0cyclictest0-21swapper/108:11:421
20709991515,0cyclictest0-21swapper/008:11:420
20709991515,0cyclictest0-21swapper/007:51:420
20723991414,0cyclictest0-21swapper/307:50:173
2072399140,0cyclictest0-21swapper/308:40:273
2072399140,0cyclictest0-21swapper/308:35:373
20719991414,0cyclictest0-21swapper/208:21:062
20719991414,0cyclictest0-21swapper/207:35:322
20712991414,0cyclictest0-21swapper/108:40:271
2071299140,0cyclictest0-21swapper/107:50:171
20709991414,0cyclictest0-21swapper/007:25:340
2070999140,14cyclictest0-21swapper/008:50:320
2070999140,0cyclictest0-21swapper/008:55:360
2070999140,0cyclictest0-21swapper/008:35:540
2070999140,0cyclictest0-21swapper/008:26:190
2070999140,0cyclictest0-21swapper/008:15:080
2070999140,0cyclictest0-21swapper/008:00:080
2070999140,0cyclictest0-21swapper/007:45:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional