You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-15 - 01:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon Sep 14, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1733899280,24cyclictest0-21swapper/312:35:313
1733899280,1cyclictest0-21swapper/308:22:173
1733899270,2cyclictest3217-21ssh10:57:313
1733899270,26cyclictest0-21swapper/312:10:023
1733899270,26cyclictest0-21swapper/311:03:083
17338992621,4cyclictest10666-21diskmemload12:29:263
17338992620,5cyclictest13983-21ssh10:38:383
17338992619,6cyclictest12531-21ssh12:32:263
1733899260,25cyclictest24913-21ssh12:15:443
1733899260,25cyclictest0-21swapper/309:12:043
1733899260,1cyclictest5077-21ssh11:57:273
1733899260,1cyclictest28852-21sh11:21:563
17338992522,3cyclictest0-21swapper/311:06:153
17338992521,3cyclictest0-21swapper/311:37:163
17338992520,4cyclictest1313-21ssh09:29:573
1733899250,24cyclictest3509-21ssh10:29:313
1733899250,24cyclictest10666-21diskmemload10:17:193
1733899250,24cyclictest0-21swapper/312:00:393
1733899250,24cyclictest0-21swapper/311:44:593
1733899250,21cyclictest0-21swapper/311:48:233
1733899250,1cyclictest1451-21munin-run11:55:003
1733899250,1cyclictest0-21swapper/310:00:213
1733899250,1cyclictest0-21swapper/309:55:433
1733899250,0cyclictest0-21swapper/310:11:113
1733899250,0cyclictest0-21swapper/310:08:153
1733799250,24cyclictest29382-21ssh10:23:072
1733799250,24cyclictest10666-21diskmemload11:27:412
17332992524,1cyclictest0-21swapper/010:33:130
1733299250,24cyclictest10666-21diskmemload09:17:150
1733299250,24cyclictest0-21swapper/009:57:160
17338992420,4cyclictest0-21swapper/311:14:243
17338992420,3cyclictest0-21swapper/309:22:093
1733899240,23cyclictest0-21swapper/310:51:183
1733899240,1cyclictest606-21dbus-daemon08:12:243
1733899240,1cyclictest23480-21sed08:27:253
1733899240,0cyclictest0-21swapper/309:17:143
17337992419,4cyclictest0-21swapper/211:35:212
1733799240,24cyclictest10666-21diskmemload12:07:292
1733799240,23cyclictest10666-21diskmemload11:24:542
1733799240,23cyclictest0-21swapper/211:34:362
1733799240,23cyclictest0-21swapper/209:22:062
17336992424,0cyclictest0-21swapper/110:42:521
17336992423,1cyclictest10666-21diskmemload10:18:211
1733699240,1cyclictest7385-21ssh09:35:211
1733699240,0cyclictest0-21swapper/111:27:161
1733299242,21cyclictest62550irq/127-eno109:14:250
1733299240,24cyclictest0-21swapper/012:39:210
1733299240,24cyclictest0-21swapper/011:29:350
1733299240,23cyclictest0-21swapper/011:03:090
1733299240,0cyclictest0-21swapper/011:42:320
17338992320,3cyclictest0-21swapper/309:35:463
17338992319,4cyclictest0-21swapper/311:34:123
17338992319,3cyclictest0-21swapper/312:22:113
17338992319,3cyclictest0-21swapper/309:50:513
17338992318,4cyclictest12799-21ssh12:05:023
17338992318,4cyclictest10666-21diskmemload10:32:263
1733899230,2cyclictest29553-21latency_hist07:35:013
1733899230,23cyclictest0-21swapper/311:26:053
1733899230,23cyclictest0-21swapper/311:15:513
1733899230,23cyclictest0-21swapper/310:48:053
1733899230,23cyclictest0-21swapper/309:49:383
1733899230,1cyclictest8347-21ptp4l-jitter07:57:233
1733899230,0cyclictest0-21swapper/310:40:093
1733899230,0cyclictest0-21swapper/309:41:573
1733899230,0cyclictest0-21swapper/309:31:223
17337992323,0cyclictest2268-21apt-get07:47:162
17337992323,0cyclictest10666-21diskmemload09:57:112
17337992322,1cyclictest1363-1kworker/u9:109:33:452
1733799230,23cyclictest0-21swapper/212:17:202
1733799230,23cyclictest0-21swapper/211:59:082
1733799230,23cyclictest0-21swapper/211:49:332
1733799230,23cyclictest0-21swapper/209:18:402
1733799230,22cyclictest7562-21/usr/sbin/munin11:02:192
1733799230,22cyclictest4232-21rm10:30:172
1733799230,22cyclictest31868-21sendmail07:40:012
1733799230,22cyclictest27260-21ssh10:51:022
1733799230,22cyclictest16912-21ssh09:43:262
1733799230,22cyclictest1649-21cron08:45:002
1733799230,22cyclictest10666-21diskmemload10:35:052
1733799230,22cyclictest0-21swapper/212:34:372
1733799230,22cyclictest0-21swapper/209:10:512
1733799230,0cyclictest0-21swapper/211:52:422
1733799230,0cyclictest0-21swapper/210:59:582
17336992323,0cyclictest10666-21diskmemload10:47:191
17336992323,0cyclictest0-21swapper/112:35:531
17336992323,0cyclictest0-21swapper/112:11:571
17336992323,0cyclictest0-21swapper/111:21:311
17336992323,0cyclictest0-21swapper/110:50:431
17336992323,0cyclictest0-21swapper/110:35:251
17336992323,0cyclictest0-21swapper/109:53:491
17336992323,0cyclictest0-21swapper/109:34:551
17336992321,1cyclictest0-21swapper/112:21:451
17336992321,1cyclictest0-21swapper/109:14:431
1733699230,23cyclictest10666-21diskmemload11:45:031
1733699230,23cyclictest0-21swapper/112:30:311
1733699230,23cyclictest0-21swapper/110:10:421
1733699230,22cyclictest5011-21basename12:27:161
1733699230,22cyclictest32348-21rm09:28:041
1733699230,22cyclictest19033-21wc08:20:011
1733699230,22cyclictest19033-21wc08:20:011
1733699230,22cyclictest10666-21diskmemload11:39:011
1733699230,22cyclictest10666-21diskmemload10:20:511
1733699230,22cyclictest0-21swapper/110:32:231
1733699230,22cyclictest0-21swapper/110:07:381
1733699230,0cyclictest0-21swapper/111:03:461
17332992323,0cyclictest0-21swapper/011:58:380
17332992323,0cyclictest0-21swapper/011:47:130
17332992323,0cyclictest0-21swapper/011:38:060
17332992323,0cyclictest0-21swapper/010:23:240
17332992321,1cyclictest62550irq/127-eno110:49:330
17332992321,1cyclictest0-21swapper/012:31:340
17332992321,1cyclictest0-21swapper/010:51:130
17332992321,1cyclictest0-21swapper/010:26:230
17332992321,1cyclictest0-21swapper/009:42:330
17332992321,1cyclictest0-21swapper/009:32:250
1733299231,22cyclictest62550irq/127-eno112:16:260
1733299230,2cyclictest3577-21ssh12:25:100
1733299230,23cyclictest0-21swapper/012:08:490
1733299230,23cyclictest0-21swapper/011:05:550
1733299230,22cyclictest7502-21ssh11:31:560
1733299230,22cyclictest20988-21rm10:16:460
1733299230,22cyclictest20558-21sh11:13:350
1733299230,22cyclictest10666-21diskmemload10:58:370
1733299230,22cyclictest10666-21diskmemload09:50:200
1733299230,22cyclictest0-21swapper/010:35:070
1733299230,22cyclictest0-21swapper/010:09:070
1733299230,22cyclictest0-21swapper/009:37:170
1733299230,0cyclictest0-21swapper/011:16:010
1733299230,0cyclictest0-21swapper/010:11:010
1733899220,1cyclictest0-21swapper/308:47:173
1733899220,1cyclictest0-21swapper/307:20:013
17337992222,0cyclictest0-21swapper/212:01:472
17337992222,0cyclictest0-21swapper/211:15:262
17337992222,0cyclictest0-21swapper/210:46:302
17337992220,1cyclictest16594-21cron08:15:002
17337992218,3cyclictest0-21swapper/212:20:022
1733799220,22cyclictest0-21swapper/209:50:032
1733799220,22cyclictest0-21swapper/209:45:502
1733799220,21cyclictest4175-21ssh12:26:022
1733799220,21cyclictest3950-21munin-run10:30:002
1733799220,21cyclictest3908-21apt-get08:57:152
1733799220,21cyclictest27368-21apt-get07:32:162
1733799220,21cyclictest2707-21sh10:00:092
1733799220,21cyclictest22090-21wc07:20:002
1733799220,21cyclictest10666-21diskmemload10:41:112
1733799220,1cyclictest15131-21ssh12:35:432
1733799220,1cyclictest11960-21ssh11:06:402
1733799220,1cyclictest0-21swapper/210:06:392
1733799220,0cyclictest0-21swapper/211:12:302
17336992222,0cyclictest0-21swapper/111:11:021
17336992221,1cyclictest2425-21ssh11:56:071
17336992221,1cyclictest10666-21diskmemload12:07:461
17336992221,1cyclictest0-21swapper/109:40:251
17336992220,1cyclictest7936-21perf10:05:011
17336992220,1cyclictest6617-21awk07:55:001
17336992220,1cyclictest29332-21turbostat07:35:011
17336992220,1cyclictest29070-21ssh11:50:531
17336992220,1cyclictest0-21swapper/112:15:381
17336992220,1cyclictest0-21swapper/108:27:161
1733699220,21cyclictest7268-21ssh12:00:091
1733699220,21cyclictest31956-21irqstats09:57:191
1733699220,21cyclictest10666-21diskmemload11:05:301
1733699220,21cyclictest0-21swapper/110:55:451
1733699220,21cyclictest0-21swapper/110:25:041
1733699220,1cyclictest7341-21ssh11:31:331
1733699220,1cyclictest32285-21apt-get07:42:181
1733699220,1cyclictest17828-21ssh11:40:381
1733699220,1cyclictest0-21swapper/109:48:441
1733699220,1cyclictest0-21swapper/109:22:521
1733699220,0cyclictest0-21swapper/109:15:251
17332992222,0cyclictest10666-21diskmemload09:49:110
17332992221,1cyclictest0-21swapper/009:21:420
17332992221,0cyclictest0-21swapper/008:32:160
17332992220,1cyclictest24497-21dump-pmu-power07:25:000
17332992220,1cyclictest1080-21idleruntime-cro08:50:000
17332992219,2cyclictest3483-21wc08:55:000
1733299220,21cyclictest313-21systemd-journal08:20:010
1733299220,21cyclictest313-21systemd-journal08:20:010
1733299220,21cyclictest0-21swapper/008:37:160
1733299220,1cyclictest6415-21ssh10:02:460
1733299220,1cyclictest28984-21sh11:22:060
1733299220,1cyclictest20611-21munin-run10:45:010
1733299220,1cyclictest0-21swapper/012:20:550
1733299220,1cyclictest0-21swapper/011:51:390
1733299220,1cyclictest0-21swapper/009:25:210
1733299220,0cyclictest0-21swapper/012:10:080
1733299220,0cyclictest0-21swapper/012:02:170
17338992118,3cyclictest0-21swapper/308:17:163
17338992118,3cyclictest0-21swapper/308:17:163
1733899210,2cyclictest25379-21ssh10:20:063
1733899210,1cyclictest28719-21grep08:40:003
1733899210,1cyclictest0-21swapper/307:55:003
1733899210,0cyclictest0-21swapper/309:02:153
1733899210,0cyclictest0-21swapper/307:42:173
17337992121,0cyclictest0-21swapper/209:36:012
17337992121,0cyclictest0-21swapper/208:22:172
1733799210,21cyclictest0-21swapper/210:10:132
1733799210,0cyclictest1501-21apt-get08:52:152
1733799210,0cyclictest10666-21diskmemload12:10:182
17336992121,0cyclictest29761-21apt-get07:37:151
17336992119,1cyclictest0-21swapper/109:05:001
1733699210,0cyclictest0-21swapper/108:32:161
17332992121,0cyclictest0-21swapper/009:02:150
17332992119,1cyclictest23847-21awk08:30:010
1733299210,1cyclictest11433-21apt-get08:05:020
17338992020,0cyclictest0-21swapper/308:42:173
17338992020,0cyclictest0-21swapper/308:02:163
17338992020,0cyclictest0-21swapper/307:12:153
1733899200,0cyclictest0-21swapper/307:47:163
17337992020,0cyclictest0-21swapper/208:47:152
17337992019,1cyclictest0-21swapper/208:00:002
1733699200,20cyclictest0-21swapper/108:52:151
1733699200,0cyclictest0-21swapper/109:07:171
1733699200,0cyclictest0-21swapper/108:42:171
17332992020,0cyclictest0-21swapper/007:37:150
17338991919,0cyclictest0-21swapper/308:57:153
17338991917,1cyclictest10666-21diskmemload09:10:013
17338991913,5cyclictest0-21swapper/307:37:153
1733899190,0cyclictest0-21swapper/308:52:153
1733899190,0cyclictest0-21swapper/308:32:173
17337991919,0cyclictest0-21swapper/207:52:152
17337991918,1cyclictest0-21swapper/208:17:162
17337991918,1cyclictest0-21swapper/208:17:162
17336991917,1cyclictest22104-21sendmail-msp07:20:001
17332991919,0cyclictest0-21swapper/007:52:160
17332991919,0cyclictest0-21swapper/007:17:160
17332991917,1cyclictest1813-21cron07:45:010
17338991818,0cyclictest0-21swapper/307:22:163
1733899180,17cyclictest0-21swapper/308:07:233
17337991818,0cyclictest8721-21apt-get09:07:172
17337991816,1cyclictest26939-21awk07:30:012
17337991816,1cyclictest19691-21munin-run07:15:002
17336991818,0cyclictest0-21swapper/107:27:161
17332991818,0cyclictest0-21swapper/007:12:150
1733899170,0cyclictest0-21swapper/307:27:163
1733299178,9cyclictest30303-21kworker/0:107:57:000
17337991613,2cyclictest28685-21ls08:40:002
17336991614,1cyclictest5905-21sendmail-msp09:00:001
1733699160,15cyclictest20893-21cat08:22:251
17332991613,3cyclictest26311-21kworker/0:208:44:480
17332991612,4cyclictest30303-21kworker/0:107:32:120
1733799150,1cyclictest0-21swapper/208:27:172
17336991515,0cyclictest17709-21apt-get07:12:151
17336991513,2cyclictest606-21dbus-daemon08:50:001
1733699150,1cyclictest0-21swapper/107:57:231
1733299150,14cyclictest0-21swapper/007:26:320
1733799140,14cyclictest0-21swapper/209:00:542
1733799140,14cyclictest0-21swapper/208:30:072
1733799140,14cyclictest0-21swapper/208:02:022
1733799140,14cyclictest0-21swapper/207:20:112
1733799140,0cyclictest0-21swapper/208:05:102
1733799140,0cyclictest0-21swapper/207:41:512
17336991414,0cyclictest0-21swapper/108:10:081
17336991414,0cyclictest0-21swapper/108:05:281
17336991414,0cyclictest0-21swapper/108:00:081
1733699140,14cyclictest0-21swapper/107:45:131
1733699140,14cyclictest0-21swapper/107:20:131
17332991414,0cyclictest0-21swapper/009:05:010
17332991414,0cyclictest0-21swapper/007:46:290
1733299140,14cyclictest0-21swapper/008:00:050
1733299140,0cyclictest0-21swapper/008:55:270
1733299140,0cyclictest0-21swapper/008:20:570
1733299140,0cyclictest0-21swapper/008:10:280
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional