You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-10-01 - 19:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Thu Oct 01, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1944499270,26cyclictest32184-21ssh11:50:492
1943999270,26cyclictest0-21swapper/109:49:371
1944499260,1cyclictest13570-21diskmemload10:56:352
1944499260,0cyclictest13570-21diskmemload10:28:122
19439992622,3cyclictest30979-21diskmemload11:20:381
1943999260,25cyclictest13570-21diskmemload12:03:441
1943999260,25cyclictest0-21swapper/112:34:581
19435992624,1cyclictest0-21swapper/011:18:470
1944999253,21cyclictest34-21ksoftirqd/310:05:003
1944999250,24cyclictest13570-21diskmemload12:09:263
19444992522,2cyclictest27615-21/usr/sbin/munin11:18:412
19444992520,4cyclictest13570-21diskmemload11:34:222
1944499250,24cyclictest13570-21diskmemload09:28:272
1944499250,24cyclictest0-21swapper/211:08:282
1944499250,1cyclictest1924-21ssh10:54:262
1944499250,0cyclictest0-21swapper/212:33:142
1944499250,0cyclictest0-21swapper/210:31:182
19439992521,3cyclictest0-21swapper/112:36:051
1943999250,24cyclictest0-21swapper/110:43:421
1943999250,21cyclictest0-21swapper/112:14:461
1943999250,21cyclictest0-21swapper/109:58:571
1943999250,0cyclictest0-21swapper/110:29:441
1943999250,0cyclictest0-21swapper/107:23:401
19435992523,1cyclictest0-21swapper/010:19:570
19435992522,2cyclictest0-21swapper/010:28:380
1943599250,24cyclictest0-21swapper/009:14:160
19449992423,1cyclictest34-21ksoftirqd/309:51:123
19449992422,1cyclictest34-21ksoftirqd/311:02:133
1944999240,24cyclictest34-21ksoftirqd/310:17:203
1944999240,23cyclictest13570-21diskmemload10:43:123
1944999240,1cyclictest606-21dbus-daemon09:20:003
19444992422,1cyclictest10297-21ssh12:00:072
19444992421,3cyclictest0-21swapper/211:59:012
19444992420,4cyclictest0-21swapper/210:19:222
19444992420,3cyclictest0-21swapper/209:31:262
19444992420,1cyclictest13021-21ssh10:35:342
1944499241,1cyclictest0-21swapper/211:03:412
1944499240,24cyclictest0-21swapper/209:10:032
1944499240,23cyclictest27875-21ssh09:22:122
1944499240,23cyclictest21374-21sh11:12:452
1944499240,23cyclictest0-21swapper/212:25:242
1944499240,23cyclictest0-21swapper/211:46:422
1944499240,23cyclictest0-21swapper/211:20:422
1944499240,23cyclictest0-21swapper/209:37:272
1944499240,20cyclictest31751-21apt-get08:43:392
1944499240,1cyclictest13773-21ssh10:07:522
1944499240,0cyclictest0-21swapper/211:26:532
1943999240,24cyclictest0-21swapper/109:22:101
1943999240,23cyclictest32612-21sh09:26:181
1943999240,23cyclictest18776-21runrttasks09:43:401
1943999240,1cyclictest0-21swapper/111:08:351
1943999240,0cyclictest17255-21apt-get09:13:411
1943999240,0cyclictest13570-21diskmemload10:36:271
1943999240,0cyclictest0-21swapper/111:25:111
1943999240,0cyclictest0-21swapper/110:18:551
1943999240,0cyclictest0-21swapper/109:32:591
1943999240,0cyclictest0-21swapper/109:17:401
19435992424,0cyclictest0-21swapper/010:57:260
19435992423,1cyclictest13570-21diskmemload10:33:220
1943599242,21cyclictest62550irq/127-eno110:41:090
19435992422,1cyclictest0-21swapper/011:38:060
19435992421,2cyclictest12758-21sh11:04:240
19435992421,2cyclictest0-21swapper/009:21:320
19435992420,3cyclictest17686-21sh09:42:110
1943599240,24cyclictest0-21swapper/012:12:260
1943599240,24cyclictest0-21swapper/009:32:540
1943599240,23cyclictest5632-21turbostat.cron10:00:000
1943599240,23cyclictest13570-21diskmemload10:20:250
1943599240,23cyclictest0-21swapper/012:05:390
1943599240,0cyclictest13570-21diskmemload11:12:540
19449992323,0cyclictest13570-21diskmemload11:45:463
19449992323,0cyclictest13570-21diskmemload11:43:113
19449992323,0cyclictest13570-21diskmemload10:03:473
19449992323,0cyclictest0-21swapper/311:09:193
19449992323,0cyclictest0-21swapper/309:45:213
19449992322,1cyclictest34-21ksoftirqd/312:28:513
19449992322,1cyclictest13066-21ssh10:35:383
19449992320,2cyclictest27125-21sed10:48:443
1944999230,2cyclictest27903-21sh12:16:443
1944999230,23cyclictest11691-21apt-get08:03:393
1944999230,23cyclictest11691-21apt-get08:03:393
1944999230,23cyclictest0-21swapper/311:33:593
1944999230,23cyclictest0-21swapper/310:21:023
1944999230,0cyclictest13570-21diskmemload11:35:473
1944999230,0cyclictest0-21swapper/312:11:123
1944999230,0cyclictest0-21swapper/309:58:213
19444992323,0cyclictest0-21swapper/211:40:402
19444992323,0cyclictest0-21swapper/209:46:202
19444992320,3cyclictest0-21swapper/212:19:182
19444992320,2cyclictest13570-21diskmemload12:20:432
19444992320,2cyclictest0-21swapper/212:37:352
19444992319,3cyclictest0-21swapper/210:23:462
1944499230,22cyclictest6468-21ssh10:00:462
1944499230,22cyclictest16456-21ssh09:40:162
1944499230,22cyclictest1192-21ssh09:56:042
1944499230,20cyclictest0-21swapper/212:14:342
1944499230,0cyclictest4530-21apt-get08:53:412
1944499230,0cyclictest0-21swapper/210:40:502
1944499230,0cyclictest0-21swapper/209:53:022
1944499230,0cyclictest0-21swapper/209:15:562
19439992323,0cyclictest0-21swapper/110:55:381
19439992322,1cyclictest32339-21sh12:20:091
19439992321,2cyclictest0-21swapper/111:52:081
19439992321,2cyclictest0-21swapper/111:47:151
19439992321,1cyclictest0-21swapper/112:15:481
19439992321,1cyclictest0-21swapper/110:53:131
19439992320,3cyclictest5658-21ssh11:56:271
19439992319,4cyclictest13570-21diskmemload10:10:031
1943999230,23cyclictest0-21swapper/110:47:021
1943999230,23cyclictest0-21swapper/110:09:511
1943999230,22cyclictest29634-21sh10:21:361
1943999230,22cyclictest10513-21ssh11:03:361
1943999230,22cyclictest0-21swapper/111:11:371
1943999230,22cyclictest0-21swapper/110:30:281
1943999230,22cyclictest0-21swapper/109:35:151
1943999230,20cyclictest0-21swapper/110:00:411
19435992323,0cyclictest13570-21diskmemload10:00:150
19435992323,0cyclictest0-21swapper/010:13:280
19435992323,0cyclictest0-21swapper/009:53:480
19435992321,1cyclictest0-21swapper/011:59:070
19435992321,1cyclictest0-21swapper/009:46:350
1943599230,2cyclictest28794-21basename08:35:000
1943599230,23cyclictest0-21swapper/012:02:220
1943599230,23cyclictest0-21swapper/011:40:010
1943599230,22cyclictest4613-21sh11:26:300
1943599230,22cyclictest0-21swapper/010:35:080
1943599230,22cyclictest0-21swapper/009:27:030
1943599230,1cyclictest13570-21diskmemload11:52:130
1943599230,0cyclictest0-21swapper/012:16:560
1943599230,0cyclictest0-21swapper/011:47:100
19449992222,0cyclictest13570-21diskmemload11:25:363
19449992222,0cyclictest0-21swapper/312:00:353
19449992222,0cyclictest0-21swapper/310:25:183
19449992222,0cyclictest0-21swapper/309:37:593
19449992220,1cyclictest606-21dbus-daemon12:40:003
19449992220,1cyclictest34-21ksoftirqd/307:43:403
1944999220,22cyclictest13570-21diskmemload11:11:473
1944999220,22cyclictest0-21swapper/309:10:213
1944999220,21cyclictest5576-21sh11:56:203
1944999220,21cyclictest11300-21sh12:30:443
1944999220,21cyclictest0-21swapper/310:55:283
1944999220,1cyclictest30575-21ssh10:51:463
1944999220,1cyclictest17592-21ssh10:10:233
1944999220,0cyclictest13570-21diskmemload12:21:203
1944999220,0cyclictest13570-21diskmemload11:22:343
1944999220,0cyclictest0-21swapper/309:43:503
1944999220,0cyclictest0-21swapper/309:32:593
1944999220,0cyclictest0-21swapper/309:20:103
1944999220,0cyclictest0-21swapper/308:43:393
19444992222,0cyclictest0-21swapper/210:11:522
19444992220,1cyclictest13543-21basename08:05:012
19444992220,1cyclictest13543-21basename08:05:012
19444992219,3cyclictest0-21swapper/211:36:132
1944499220,22cyclictest24415-21ssh10:45:512
1944499220,1cyclictest17897-21sh12:08:202
19439992222,0cyclictest0-21swapper/112:26:281
19439992222,0cyclictest0-21swapper/111:31:491
19439992222,0cyclictest0-21swapper/108:53:411
19439992221,1cyclictest15770-21ssh12:05:181
1943999220,21cyclictest6884-21apt-get07:53:401
1943999220,21cyclictest0-21swapper/111:15:101
1943999220,21cyclictest0-21swapper/109:53:481
1943999220,1cyclictest0-21swapper/111:44:041
1943999220,0cyclictest0-21swapper/111:35:221
19435992222,0cyclictest0-21swapper/012:30:490
19435992222,0cyclictest0-21swapper/010:54:270
19435992221,1cyclictest10369-21ssh09:35:010
19435992221,1cyclictest0-21swapper/011:20:200
19435992221,1cyclictest0-21swapper/009:16:590
19435992220,1cyclictest31453-21munin-run07:35:000
1943599220,21cyclictest3937-21awk07:45:000
1943599220,21cyclictest0-21swapper/012:36:200
1943599220,21cyclictest0-21swapper/011:05:470
1943599220,1cyclictest3592-21apt-get08:50:020
1943599220,1cyclictest0-21swapper/012:29:240
1943599220,1cyclictest0-21swapper/011:30:300
1943599220,1cyclictest0-21swapper/008:10:010
1943599220,0cyclictest0-21swapper/010:46:570
19449992121,0cyclictest0-21swapper/310:33:343
19449992121,0cyclictest0-21swapper/308:33:413
19449992120,1cyclictest31903-21ssh11:50:273
19449992120,1cyclictest19274-21apt-get08:18:393
1944999210,20cyclictest31211-21sendmail08:40:003
1944999210,20cyclictest25153-21ssh11:15:203
1944999210,20cyclictest0-21swapper/307:58:433
1944999210,20cyclictest0-21swapper/307:48:403
1944999210,1cyclictest0-21swapper/308:58:393
1944999210,0cyclictest14088-21apt-get08:08:413
1944999210,0cyclictest0-21swapper/309:25:253
19444992120,1cyclictest24742-21apt-get07:23:402
19444992119,1cyclictest21170-21sendmail-msp08:20:012
1944499210,20cyclictest0-21swapper/208:08:442
1944499210,0cyclictest0-21swapper/209:03:422
19439992120,1cyclictest13563-21perf08:05:011
19439992120,1cyclictest13563-21perf08:05:011
19439992120,1cyclictest1201-21perf08:45:001
1943999210,0cyclictest0-21swapper/108:00:001
19435992121,0cyclictest0-21swapper/007:38:400
1943599210,20cyclictest0-21swapper/007:48:400
1943599210,20cyclictest0-21swapper/007:18:410
1943599210,1cyclictest0-21swapper/008:10:020
1943599210,0cyclictest0-21swapper/007:58:420
19449992020,0cyclictest29585-21apt-get07:33:413
19449992020,0cyclictest0-21swapper/307:18:413
19449992019,1cyclictest34-21ksoftirqd/308:30:003
1944999200,20cyclictest0-21swapper/307:13:393
1944999200,0cyclictest0-21swapper/307:23:403
19444992017,2cyclictest13584-21latency_hist09:10:012
19444992016,3cyclictest24488-21apt-get08:28:392
1944499200,20cyclictest0-21swapper/207:38:402
19439992020,0cyclictest11734-21apt-get09:08:421
19439992020,0cyclictest0-21swapper/107:43:401
19439992020,0cyclictest0-21swapper/107:15:011
19439992018,1cyclictest0-21swapper/108:08:441
1943999200,20cyclictest0-21swapper/108:13:401
19435992020,0cyclictest0-21swapper/008:58:390
19435992020,0cyclictest0-21swapper/008:38:410
19435992020,0cyclictest0-21swapper/008:28:390
19435992018,1cyclictest606-21dbus-daemon09:05:010
19435992018,1cyclictest19274-21apt-get08:18:390
19449991919,0cyclictest6884-21apt-get07:53:413
19449991919,0cyclictest0-21swapper/308:53:413
19449991919,0cyclictest0-21swapper/307:38:403
19449991918,1cyclictest16876-21apt-get08:13:403
19444991919,0cyclictest0-21swapper/208:58:392
19444991919,0cyclictest0-21swapper/207:13:392
19444991917,1cyclictest24209-21munin-run07:20:012
1944499190,0cyclictest9280-21apt-get07:58:432
19439991919,0cyclictest0-21swapper/107:18:411
19435991919,0cyclictest0-21swapper/007:28:400
19435991919,0cyclictest0-21swapper/007:23:400
19435991917,1cyclictest0-21swapper/008:50:000
19449991818,0cyclictest0-21swapper/309:03:423
19449991818,0cyclictest0-21swapper/308:48:393
19449991818,0cyclictest0-21swapper/307:28:403
19449991816,1cyclictest13593-21perf09:10:013
19439991818,0cyclictest0-21swapper/109:03:421
19439991817,0cyclictest31980-21apt-get07:38:401
1943999180,17cyclictest6335-21ls07:50:001
19435991818,0cyclictest0-21swapper/008:43:400
19444991717,0cyclictest0-21swapper/207:48:402
19444991716,1cyclictest30389-21irqstats07:33:442
19444991716,0cyclictest26889-21apt-get08:33:412
19439991717,0cyclictest0-21swapper/108:28:391
19439991715,1cyclictest21171-21perf08:20:011
1943999170,0cyclictest0-21swapper/107:28:401
1943599170,16cyclictest0-21swapper/009:08:420
1944999160,15cyclictest22255-21perl08:23:443
1944499160,16cyclictest0-21swapper/208:46:192
1944499160,14cyclictest0-21swapper/208:14:022
19435991614,1cyclictest0-21swapper/007:53:420
19444991515,0cyclictest0-21swapper/208:38:412
19444991515,0cyclictest0-21swapper/207:43:402
19444991513,0cyclictest0-21swapper/207:26:142
1944499150,15cyclictest0-21swapper/207:53:592
19439991515,0cyclictest0-21swapper/108:58:391
19439991514,1cyclictest0-21swapper/108:30:591
19439991514,1cyclictest0-21swapper/108:24:471
19439991514,1cyclictest0-21swapper/107:30:491
1944499140,14cyclictest0-21swapper/208:20:102
19439991414,0cyclictest0-21swapper/108:45:561
19439991414,0cyclictest0-21swapper/108:35:311
19435991414,0cyclictest0-21swapper/008:21:120
1943599140,14cyclictest0-21swapper/008:00:320
1943599140,14cyclictest0-21swapper/008:00:320
1943599140,14cyclictest0-21swapper/007:12:410
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional