You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-20 - 18:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Jun 20, 2026 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5092992820,4cyclictest31255-21diskmemload12:18:382
509299280,22cyclictest18354-21idleruntime-cro11:30:012
5092992723,4cyclictest0-21swapper/209:43:152
5092992720,3cyclictest0-21swapper/210:37:272
5092992719,7cyclictest24119-21latency_hist11:05:002
509299270,21cyclictest0-21swapper/210:34:302
508499270,0cyclictest20705-21apt-get10:00:110
5092992622,4cyclictest0-21swapper/210:25:122
5092992621,4cyclictest31255-21diskmemload12:04:542
509299260,25cyclictest0-21swapper/210:05:092
509299260,25cyclictest0-21swapper/209:26:412
509299260,1cyclictest0-21swapper/211:58:462
509299260,1cyclictest0-21swapper/211:58:462
509299260,0cyclictest0-21swapper/212:25:102
508499260,25cyclictest24994-21ssh11:35:170
508499260,1cyclictest31255-21diskmemload09:39:100
5096992522,2cyclictest22013-21idleruntime-cro07:45:003
509699250,24cyclictest31255-21diskmemload11:55:553
509699250,24cyclictest31255-21diskmemload11:55:553
5092992522,3cyclictest0-21swapper/212:24:572
5092992518,6cyclictest25620-21turbostat10:05:002
509299250,24cyclictest0-21swapper/212:14:072
509299250,24cyclictest0-21swapper/209:37:112
509299250,20cyclictest0-21swapper/210:12:572
5088992523,1cyclictest0-21swapper/111:47:031
5084992522,2cyclictest8094-21ssh09:18:050
508499250,24cyclictest75-21kswapd009:45:020
508499250,24cyclictest0-21swapper/012:20:300
508499250,21cyclictest21035-21ssh11:00:260
508499250,1cyclictest3945-21rm12:16:380
508499250,1cyclictest0-21swapper/012:00:150
508499250,0cyclictest31255-21diskmemload11:52:320
508499250,0cyclictest26009-21apt-get10:05:120
509699240,23cyclictest624-21ssh11:12:083
509699240,23cyclictest31255-21diskmemload11:44:193
509699240,23cyclictest31255-21diskmemload10:22:423
509699240,23cyclictest31255-21diskmemload10:00:483
509699240,23cyclictest1526-21ssh09:11:043
5092992422,1cyclictest0-21swapper/209:23:512
5092992419,4cyclictest5166-21ssh11:15:562
5092992419,4cyclictest0-21swapper/212:31:142
5092992419,4cyclictest0-21swapper/211:31:202
5092992418,5cyclictest0-21swapper/209:30:132
509299240,20cyclictest0-21swapper/209:58:512
509299240,20cyclictest0-21swapper/209:46:212
509299240,0cyclictest0-21swapper/212:06:472
509299240,0cyclictest0-21swapper/211:09:512
5088992421,2cyclictest9198-21ssh12:21:411
508899240,23cyclictest2662-21ssh10:13:111
508899240,23cyclictest21475-21ssh12:02:531
508899240,23cyclictest15089-21ssh11:56:071
508899240,23cyclictest15089-21ssh11:56:071
508899240,23cyclictest0-21swapper/112:32:241
508899240,23cyclictest0-21swapper/110:02:461
508899240,23cyclictest0-21swapper/109:33:571
508899240,1cyclictest31255-21diskmemload09:35:381
508899240,0cyclictest31255-21diskmemload10:36:501
5084992423,1cyclictest62550irq/127-eno111:29:070
5084992423,1cyclictest0-21swapper/010:12:140
5084992422,2cyclictest22059-21ssh10:30:430
5084992422,2cyclictest0-21swapper/009:28:270
508499242,21cyclictest18621-21rm09:57:190
5084992421,3cyclictest0-21swapper/010:23:090
5084992420,4cyclictest31255-21diskmemload10:41:540
5084992420,3cyclictest31255-21diskmemload09:30:580
5084992420,3cyclictest13470-21ssh09:23:120
5084992419,5cyclictest0-21swapper/012:25:100
508499240,24cyclictest0-21swapper/010:52:500
508499240,23cyclictest31255-21diskmemload10:17:070
508499240,21cyclictest31255-21diskmemload12:09:440
508499240,21cyclictest0-21swapper/011:11:260
508499240,20cyclictest0-21swapper/011:57:130
508499240,20cyclictest0-21swapper/011:57:130
5096992323,0cyclictest0-21swapper/311:51:413
5096992323,0cyclictest0-21swapper/311:39:413
5096992323,0cyclictest0-21swapper/311:27:363
5096992323,0cyclictest0-21swapper/311:22:383
5096992323,0cyclictest0-21swapper/310:34:453
5096992322,1cyclictest75-21kswapd010:15:123
5096992321,1cyclictest5565-21ssh11:47:433
5096992321,1cyclictest10932-21ssh10:51:063
5096992321,1cyclictest0-21swapper/312:35:163
5096992320,2cyclictest16571-21turbostat.cron12:30:003
509699230,2cyclictest18671-21ssh11:30:113
509699230,23cyclictest31255-21diskmemload09:15:143
509699230,22cyclictest947-21munin-run12:15:013
509699230,22cyclictest31359-21ssh09:40:133
509699230,22cyclictest31255-21diskmemload09:30:213
509699230,22cyclictest27573-21sh10:05:213
509699230,22cyclictest16495-21awk08:40:003
509699230,0cyclictest31255-21diskmemload09:47:113
509699230,0cyclictest0-21swapper/310:44:173
509699230,0cyclictest0-21swapper/310:27:093
5092992323,0cyclictest31255-21diskmemload12:37:492
5092992322,1cyclictest31255-21diskmemload10:17:562
5092992320,3cyclictest0-21swapper/211:42:032
5092992320,3cyclictest0-21swapper/209:15:482
5092992319,4cyclictest0-21swapper/210:50:122
5092992318,1cyclictest0-21swapper/211:46:322
5092992317,1cyclictest0-21swapper/210:46:362
509299230,23cyclictest0-21swapper/211:53:162
509299230,23cyclictest0-21swapper/209:51:512
509299230,22cyclictest32024-21apt08:05:012
509299230,22cyclictest0-21swapper/211:12:192
509299230,22cyclictest0-21swapper/210:20:292
509299230,22cyclictest0-21swapper/208:55:012
509299230,0cyclictest31255-21diskmemload11:35:532
509299230,0cyclictest31255-21diskmemload10:43:132
5088992323,0cyclictest0-21swapper/110:40:111
5088992323,0cyclictest0-21swapper/109:45:091
5088992323,0cyclictest0-21swapper/109:41:311
5088992323,0cyclictest0-21swapper/107:40:121
5088992321,1cyclictest310-21ntp_states09:10:161
5088992321,1cyclictest28727-21sh10:06:401
5088992321,1cyclictest14984-21apt-get07:30:111
5088992321,1cyclictest14682-21apt-get09:55:021
5088992321,1cyclictest0-21swapper/112:05:441
5088992321,1cyclictest0-21swapper/111:06:361
5088992321,1cyclictest0-21swapper/110:53:231
5088992321,1cyclictest0-21swapper/109:25:161
5088992320,2cyclictest23769-21rm10:33:221
508899230,23cyclictest0-21swapper/111:50:361
508899230,23cyclictest0-21swapper/111:26:051
508899230,22cyclictest11395-21timerandwakeup09:20:191
508899230,22cyclictest0-21swapper/112:39:581
508899230,22cyclictest0-21swapper/110:45:111
508899230,22cyclictest0-21swapper/109:50:191
508899230,22cyclictest0-21swapper/108:55:111
508899230,1cyclictest6814-21ssh10:16:281
508899230,1cyclictest17180-21ssh10:26:121
508899230,1cyclictest10141-21apt-get07:20:111
508899230,1cyclictest0-21swapper/112:14:361
508899230,1cyclictest0-21swapper/111:12:251
508899230,0cyclictest31255-21diskmemload10:20:431
5084992323,0cyclictest0-21swapper/009:14:350
5084992322,1cyclictest1043-21ssh11:43:590
5084992320,3cyclictest0-21swapper/012:30:040
5084992320,3cyclictest0-21swapper/011:16:050
508499230,23cyclictest31255-21diskmemload12:37:470
508499230,23cyclictest0-21swapper/010:26:520
508499230,22cyclictest17348-21seq10:57:490
508499230,0cyclictest31255-21diskmemload11:23:500
508499230,0cyclictest0-21swapper/010:37:120
508499230,0cyclictest0-21swapper/009:52:230
5096992222,0cyclictest0-21swapper/312:02:503
5096992222,0cyclictest0-21swapper/310:35:543
5096992222,0cyclictest0-21swapper/309:55:193
5096992220,1cyclictest31255-21diskmemload12:16:363
509699220,2cyclictest28977-21ssh11:09:013
509699220,22cyclictest15801-21ssh10:55:313
509699220,22cyclictest0-21swapper/312:33:073
509699220,21cyclictest7099-21apt-get08:20:103
509699220,21cyclictest31255-21diskmemload10:10:223
509699220,21cyclictest27482-21apt-get07:55:113
509699220,21cyclictest19986-21ssh11:00:153
509699220,21cyclictest19146-21apt-get08:45:113
509699220,21cyclictest19146-21apt-get08:45:113
509699220,21cyclictest10441-21diskmemload09:20:133
509699220,21cyclictest0-21swapper/312:20:123
509699220,21cyclictest0-21swapper/310:46:073
509699220,1cyclictest19024-21ssh09:28:323
509699220,0cyclictest0-21swapper/312:09:343
509699220,0cyclictest0-21swapper/311:17:073
5092992222,0cyclictest31255-21diskmemload10:57:272
5092992222,0cyclictest0-21swapper/209:13:202
5092992221,1cyclictest32294-21apt-get08:05:112
5092992220,1cyclictest0-21swapper/208:25:002
509299221,20cyclictest18881-21latency_hist08:45:002
509299220,1cyclictest0-21swapper/208:50:012
509299220,1cyclictest0-21swapper/208:50:012
5088992222,0cyclictest31255-21diskmemload11:33:041
5088992222,0cyclictest31255-21diskmemload11:23:561
5088992222,0cyclictest0-21swapper/109:15:461
5088992220,1cyclictest7468-21perf07:15:011
5088992220,1cyclictest7468-21perf07:15:011
5088992220,1cyclictest31255-21diskmemload11:00:191
508899220,21cyclictest27509-21sh11:37:531
508899220,21cyclictest18893-21munin-run08:45:001
508899220,21cyclictest14597-21if_err_enp1s010:55:161
508899220,21cyclictest0-21swapper/112:25:071
508899220,21cyclictest0-21swapper/108:20:101
508899220,1cyclictest31257-21turbostat09:10:001
508899220,1cyclictest31255-21diskmemload12:16:171
508899220,0cyclictest31255-21diskmemload11:18:281
508899220,0cyclictest0-21swapper/111:40:061
5084992220,2cyclictest5366-21ssh10:45:370
5084992220,1cyclictest606-21dbus-daemon07:45:000
5084992220,1cyclictest606-21dbus-daemon07:40:000
5084992220,1cyclictest29601-21awk08:00:010
5084992220,1cyclictest2124-21ssh11:45:060
5084992219,2cyclictest19924-21ssh11:30:190
5084992216,5cyclictest29758-21munin-run11:10:010
508499220,22cyclictest0-21swapper/008:00:010
508499220,21cyclictest1078-21munin-run12:15:010
508499220,21cyclictest0-21swapper/008:25:130
508499220,1cyclictest1367-21ssh09:41:120
5096992121,0cyclictest0-21swapper/308:15:113
5096992121,0cyclictest0-21swapper/307:25:123
5096992120,1cyclictest17381-21apt-get07:35:133
509699210,21cyclictest0-21swapper/309:50:113
509699210,20cyclictest0-21swapper/309:00:113
509699210,20cyclictest0-21swapper/307:20:113
509699210,0cyclictest0-21swapper/307:30:113
5092992119,1cyclictest12259-21cron07:25:012
509299210,20cyclictest0-21swapper/207:15:112
5088992121,0cyclictest0-21swapper/108:45:111
5088992121,0cyclictest0-21swapper/108:45:111
5084992116,4cyclictest31282-21munin-run09:10:000
508499210,20cyclictest0-21swapper/007:45:110
508499210,0cyclictest0-21swapper/008:15:110
5096992020,0cyclictest0-21swapper/308:30:103
5096992020,0cyclictest0-21swapper/308:10:123
5096992018,1cyclictest2025-21munin-run08:10:013
5096992017,2cyclictest7457-21grep07:15:013
5096992017,2cyclictest7457-21grep07:15:013
509699200,0cyclictest0-21swapper/308:40:113
5092992020,0cyclictest0-21swapper/207:55:112
5092992019,1cyclictest24331-21apt-get08:55:112
5092992019,1cyclictest17122-21awk07:35:002
5088992020,0cyclictest0-21swapper/107:25:121
508899200,20cyclictest0-21swapper/108:00:131
508899200,0cyclictest4674-21apt-get08:15:111
5084992015,4cyclictest0-21swapper/007:25:120
508499200,1cyclictest21286-21munin-plugin-st08:50:010
508499200,1cyclictest21286-21munin-plugin-st08:50:010
5092991919,0cyclictest9497-21apt-get08:25:132
5092991919,0cyclictest19900-21apt-get07:40:122
5092991919,0cyclictest0-21swapper/207:35:122
5092991919,0cyclictest0-21swapper/207:10:112
5092991919,0cyclictest0-21swapper/207:10:112
5088991919,0cyclictest0-21swapper/108:30:101
5088991919,0cyclictest0-21swapper/108:25:131
5088991917,1cyclictest7724-21apt-get07:15:111
5088991917,1cyclictest0-21swapper/108:05:111
508499190,3cyclictest0-21swapper/008:30:190
508499190,1cyclictest6378-21sh07:10:170
508499190,1cyclictest6378-21sh07:10:170
508499190,18cyclictest0-21swapper/008:55:110
5096991817,1cyclictest25083-21apt-get07:50:113
5092991818,0cyclictest11886-21apt-get08:30:102
5088991817,1cyclictest0-21swapper/108:15:011
5084991818,0cyclictest0-21swapper/008:20:100
5084991818,0cyclictest0-21swapper/007:20:110
5092991717,0cyclictest0-21swapper/208:15:112
5092991717,0cyclictest0-21swapper/207:45:112
5088991715,1cyclictest19612-21turbostat07:40:001
508499170,1cyclictest16060-21users08:35:220
508499170,17cyclictest0-21swapper/008:10:120
5096991614,1cyclictest26450-21cron09:00:013
509699160,1cyclictest9837-21cron07:20:013
509699160,1cyclictest11765-21munin-run08:30:003
5092991616,0cyclictest0-21swapper/209:05:122
5092991616,0cyclictest0-21swapper/207:50:112
5092991614,1cyclictest0-21swapper/208:35:162
508499166,9cyclictest1374-21/usr/sbin/munin08:05:200
508499160,0cyclictest0-21swapper/009:01:280
5096991515,0cyclictest22295-21apt-get07:45:113
5092991515,0cyclictest12537-21apt-get07:25:122
5092991514,1cyclictest0-21swapper/208:10:122
5088991515,0cyclictest0-21swapper/108:00:011
5088991514,1cyclictest24418-21apt-get07:50:021
508899150,14cyclictest0-21swapper/108:35:121
508499151,1cyclictest8413-21/usr/sbin/munin07:15:140
508499150,4cyclictest0-21swapper/008:50:110
5096991414,0cyclictest0-21swapper/308:51:493
509699140,14cyclictest0-21swapper/309:05:303
509699140,0cyclictest0-21swapper/308:01:053
5092991414,0cyclictest0-21swapper/209:00:112
5088991414,0cyclictest0-21swapper/107:45:271
508499140,14cyclictest0-21swapper/007:50:110
508499140,14cyclictest0-21swapper/007:30:480
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional