You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 05:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Sun Mar 15, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7907211993,20sleep00-21swapper/019:05:340
84692116105,6sleep10-21swapper/119:07:311
794629881,12sleep30-21swapper/319:05:533
768129683,6sleep20-21swapper/219:05:162
36372850,1sleep23643-21cpuspeed_turbos23:35:132
183462750,1sleep20-21swapper/223:16:212
28562740,0sleep2301ktimersoftd/223:34:502
38402710,1sleep0101ktimersoftd/021:23:250
38402710,1sleep0101ktimersoftd/021:23:240
275592700,0sleep20-21swapper/221:15:002
141492670,0sleep00-21swapper/023:45:180
107032670,0sleep20-21swapper/222:35:452
38182610,1sleep30-21swapper/320:20:163
8685994116,2cyclictest0-21swapper/023:20:110
868699360,18cyclictest0-21swapper/119:20:101
868899350,2cyclictest0-21swapper/319:50:113
21172290,1sleep2301ktimersoftd/221:55:012
288952260,0sleep20-21swapper/221:49:032
868899245,17cyclictest0-21swapper/300:00:123
868899232,18cyclictest0-21swapper/300:26:583
868599231,20cyclictest24030-21ssh22:50:120
868899211,18cyclictest0-21swapper/323:10:133
868599211,2cyclictest0-21swapper/022:40:090
134662210,0sleep10-21swapper/121:32:591
12642200,1sleep10-21swapper/123:00:051
8686991916,2cyclictest0-21swapper/123:15:151
868599197,3cyclictest3649-21snmpd22:05:280
868599191,16cyclictest28454-21diskmemload22:20:590
868599190,2cyclictest0-21swapper/023:09:250
868599190,17cyclictest0-21swapper/021:40:190
157052190,1sleep20-21swapper/223:46:472
8688991814,2cyclictest5359-21ssh00:10:113
868699180,2cyclictest0-21swapper/119:55:101
868599180,9cyclictest0-21swapper/021:25:450
868599180,2cyclictest0-21swapper/022:26:130
868599180,2cyclictest0-21swapper/021:54:390
868599180,2cyclictest0-21swapper/000:02:500
868599179,7cyclictest0-21swapper/000:16:130
868599178,2cyclictest0-21swapper/023:33:480
868599175,3cyclictest0-21swapper/020:40:000
868599174,4cyclictest2850-21rsyslogd20:34:330
868599174,3cyclictest3479-21aten2.4_r3power23:35:120
868599174,3cyclictest22602-21diskmemload23:26:030
868599174,3cyclictest21823-21df19:45:130
868599173,4cyclictest24803-21cat23:56:020
868599173,3cyclictest22602-21diskmemload21:31:440
868599170,9cyclictest0-21swapper/022:48:260
868599170,2cyclictest0-21swapper/000:31:370
868599170,2cyclictest0-21swapper/000:31:370
868599170,16cyclictest22602-21diskmemload00:29:120
868599170,16cyclictest0-21swapper/023:53:510
868599170,10cyclictest0-21swapper/022:37:230
868899160,3cyclictest0-21swapper/323:55:183
868899160,14cyclictest0-21swapper/321:40:173
8687991611,3cyclictest32293-21fschecks_count22:25:142
868799160,11cyclictest0-21swapper/200:32:052
868799160,11cyclictest0-21swapper/200:32:042
868699161,13cyclictest0-21swapper/123:06:021
868599168,2cyclictest0-21swapper/023:02:570
868599168,2cyclictest0-21swapper/021:58:170
868599165,2cyclictest0-21swapper/000:06:220
868599164,3cyclictest67652chrt22:31:390
868599164,3cyclictest3728-21runrttasks19:17:120
868599164,3cyclictest24989-21latency_hist19:55:000
868599164,3cyclictest22602-21diskmemload00:10:530
868599164,3cyclictest15326-21cpu21:35:120
868599164,2cyclictest382-21sshd00:37:460
868599163,3cyclictest9068-21cat22:01:020
868599163,3cyclictest8953-21cut19:10:120
868599163,3cyclictest8542-21cpu23:40:130
868599163,3cyclictest23722-21latency21:10:160
868599163,3cyclictest22865-21missed_timers22:15:180
868599163,3cyclictest22602-21diskmemload22:57:170
868599163,3cyclictest21399-21cut19:45:010
868599163,3cyclictest17266-21cpu20:55:130
868599163,3cyclictest15652-21grep00:20:150
868599163,3cyclictest11808-21fschecks_time23:10:150
868599162,4cyclictest13320-21cpuspeed_turbos20:45:120
868599162,3cyclictest17218-21proc_pri23:15:220
868599161,6cyclictest0-21swapper/021:15:170
868599161,6cyclictest0-21swapper/020:10:040
868599161,4cyclictest18326-21taskset22:10:300
868599161,4cyclictest17688-21latency_hist19:35:000
868599160,9cyclictest0-21swapper/021:45:050
868599160,6cyclictest0-21swapper/021:00:140
868599160,5cyclictest0-21swapper/020:05:040
868899152,7cyclictest0-21swapper/320:15:013
868899150,13cyclictest0-21swapper/323:40:143
8687991512,2cyclictest21310-21kworker/u8:022:34:042
8687991512,2cyclictest13364-21kworker/u8:023:59:052
8687991512,2cyclictest13102-21kworker/u8:120:51:042
868799151,12cyclictest3546-21ssh23:01:212
8687991510,3cyclictest15901-21kernelversion21:35:162
8687991510,3cyclictest12656-21diskstats22:05:122
868699151,2cyclictest5339-21apt-get21:25:131
868699151,2cyclictest0-21swapper/121:17:041
8686991512,2cyclictest29875-21kworker/u8:021:51:051
8686991510,3cyclictest2207-21grep20:16:051
868699150,2cyclictest0-21swapper/122:13:341
868699150,2cyclictest0-21swapper/121:55:541
868699150,13cyclictest18333-21grep19:35:131
868699150,13cyclictest0-21swapper/122:20:421
868699150,13cyclictest0-21swapper/122:16:031
868599154,3cyclictest3614-21latency20:20:140
868599154,2cyclictest0-21swapper/019:37:300
868599153,3cyclictest11740-21uname20:40:160
868599153,3cyclictest0-21swapper/019:20:120
868599152,3cyclictest6042-21taskset20:25:370
868599152,3cyclictest25469-21cpuspeed_turbos19:55:120
868599151,6cyclictest0-21swapper/019:29:570
868599151,4cyclictest2596-21cron20:20:000
868599151,3cyclictest15304-21tr20:50:120
868899141,7cyclictest0-21swapper/323:55:003
8688991411,2cyclictest0-21swapper/300:23:583
868899141,11cyclictest25115-21ssh22:17:463
868899140,12cyclictest0-21swapper/321:45:113
868899140,12cyclictest0-21swapper/321:32:273
868799142,2cyclictest0-21swapper/222:14:012
8687991411,2cyclictest1854-21kworker/u8:123:44:042
8687991411,2cyclictest10283-21kworker/u8:100:19:042
8687991410,2cyclictest0-21swapper/223:11:042
8687991410,2cyclictest0-21swapper/219:25:142
868799140,3cyclictest27542-21hwlatdetect22:20:142
868799140,13cyclictest0-21swapper/220:26:022
868799140,13cyclictest0-21swapper/200:10:132
868799140,12cyclictest845-21fschecks_time00:05:142
868799140,12cyclictest25126-21ssh23:24:112
868799140,12cyclictest0-21swapper/221:27:502
868799140,12cyclictest0-21swapper/220:56:032
868799140,12cyclictest0-21swapper/220:45:102
868799140,12cyclictest0-21swapper/219:10:152
868699149,3cyclictest25597-21rm23:57:001
868699141,2cyclictest9461-21perl19:10:141
868699141,2cyclictest230672sleep100:27:431
868699141,2cyclictest207150irq/24-eth100:15:131
8686991411,2cyclictest27967-21kworker/u8:021:37:041
8686991411,2cyclictest21310-21kworker/u8:022:36:041
8686991411,2cyclictest0-21swapper/123:52:041
8686991411,2cyclictest0-21swapper/123:12:041
8686991411,2cyclictest0-21swapper/122:56:141
8686991410,2cyclictest4301-21ssh00:08:551
868699140,2cyclictest0-21swapper/123:26:041
868699140,2cyclictest0-21swapper/122:45:041
868699140,13cyclictest0-21swapper/100:21:251
868699140,13cyclictest0-21swapper/100:13:341
868699140,12cyclictest9428-21entropy20:35:131
868699140,12cyclictest2609-21sh21:21:361
868699140,12cyclictest2609-21sh21:21:361
868699140,12cyclictest0-21swapper/100:30:101
868699140,12cyclictest0-21swapper/100:30:101
868599142,3cyclictest22030-21cat21:06:010
88412130,2sleep30-21swapper/322:00:573
868899132,6cyclictest0-21swapper/320:00:003
8688991310,2cyclictest0-21swapper/322:45:133
8688991310,2cyclictest0-21swapper/322:30:143
868899130,2cyclictest5332-21ssh21:57:273
868899130,2cyclictest22602-21diskmemload23:06:083
868899130,2cyclictest15857-21copy23:15:033
868899130,2cyclictest0-21swapper/323:21:383
868899130,2cyclictest0-21swapper/323:00:123
868899130,2cyclictest0-21swapper/322:40:123
868899130,2cyclictest0-21swapper/321:20:083
868899130,2cyclictest0-21swapper/321:20:083
868899130,2cyclictest0-21swapper/319:15:193
868899130,12cyclictest10230-21ssh22:35:173
868899130,12cyclictest0-21swapper/323:29:383
868899130,12cyclictest0-21swapper/320:35:173
868899130,12cyclictest0-21swapper/300:16:053
868899130,11cyclictest0-21swapper/323:37:163
868899130,11cyclictest0-21swapper/321:36:463
8687991310,2cyclictest18118-21kworker/u8:121:42:052
8687991310,2cyclictest0-21swapper/220:11:022
868799130,8cyclictest0-21swapper/223:27:042
868799130,12cyclictest0-21swapper/223:05:132
868799130,12cyclictest0-21swapper/221:05:012
868799130,11cyclictest0-21swapper/222:17:552
868699139,2cyclictest0-21swapper/120:30:201
868699138,3cyclictest32337-21taskset23:30:521
868699138,3cyclictest26665-21sh21:13:441
868699138,3cyclictest21200-21df_inode21:05:131
868699131,6cyclictest0-21swapper/119:45:001
868699131,3cyclictest0-21swapper/120:51:021
8686991310,2cyclictest0-21swapper/123:38:141
8686991310,2cyclictest0-21swapper/123:24:341
8686991310,2cyclictest0-21swapper/122:54:511
8686991310,2cyclictest0-21swapper/122:30:271
8686991310,2cyclictest0-21swapper/122:09:241
8686991310,2cyclictest0-21swapper/122:00:251
8686991310,2cyclictest0-21swapper/121:46:241
8686991310,2cyclictest0-21swapper/120:20:121
868699130,2cyclictest0-21swapper/122:26:191
868699130,2cyclictest0-21swapper/121:44:451
868699130,12cyclictest0-21swapper/120:25:131
868699130,12cyclictest0-21swapper/119:25:131
868699130,12cyclictest0-21swapper/100:35:381
868699130,12cyclictest0-21swapper/100:01:241
868899129,2cyclictest22602-21diskmemload21:25:033
868899129,2cyclictest0-21swapper/322:26:043
868899129,2cyclictest0-21swapper/322:10:113
868899129,2cyclictest0-21swapper/321:54:203
868899129,2cyclictest0-21swapper/320:55:153
868899129,2cyclictest0-21swapper/300:35:063
868899129,2cyclictest0-21swapper/300:33:023
868899129,2cyclictest0-21swapper/300:33:013
868899129,2cyclictest0-21swapper/300:05:203
868899122,7cyclictest0-21swapper/319:24:583
868899122,6cyclictest0-21swapper/319:46:013
868899121,9cyclictest31245-21ssh22:56:383
868899121,6cyclictest0-21swapper/320:35:003
868899120,2cyclictest0-21swapper/322:05:173
868899120,11cyclictest31972-21ssh23:30:203
868899120,11cyclictest0-21swapper/322:52:573
868899120,11cyclictest0-21swapper/322:20:163
868899120,11cyclictest0-21swapper/320:05:173
868899120,11cyclictest0-21swapper/319:31:013
868899120,11cyclictest0-21swapper/319:10:173
868799129,2cyclictest24904-21kworker/u8:219:58:042
868799129,2cyclictest20914-21kworker/u8:119:43:042
868799129,2cyclictest12155-21kworker/u8:119:19:042
868799129,2cyclictest0-21swapper/222:59:082
868799129,2cyclictest0-21swapper/222:51:012
868799129,2cyclictest0-21swapper/200:25:572
868799129,2cyclictest0-21swapper/200:20:152
868799120,2cyclictest31142-21ssh00:35:472
868799120,2cyclictest0-21swapper/222:43:412
868799120,2cyclictest0-21swapper/221:31:102
868799120,2cyclictest0-21swapper/221:21:112
868799120,2cyclictest0-21swapper/221:21:112
868799120,2cyclictest0-21swapper/220:30:132
868799120,2cyclictest0-21swapper/220:20:152
868799120,2cyclictest0-21swapper/219:20:162
868799120,1cyclictest15808-21apt-get19:30:122
868799120,11cyclictest0-21swapper/223:51:102
868799120,11cyclictest0-21swapper/222:00:402
868799120,11cyclictest0-21swapper/221:58:112
868799120,11cyclictest0-21swapper/221:15:522
868799120,11cyclictest0-21swapper/219:35:392
868699129,2cyclictest1854-21kworker/u8:123:45:041
868699129,2cyclictest12155-21kworker/u8:119:30:041
868699129,2cyclictest0-21swapper/122:40:151
868699129,2cyclictest0-21swapper/120:00:181
868699120,2cyclictest8343-21ssh23:40:091
868699120,2cyclictest0-21swapper/120:10:381
868699120,10cyclictest0-21swapper/121:00:401
294712120,2sleep329553-21munin-node3
868899118,2cyclictest0-21swapper/323:45:063
868899118,2cyclictest0-21swapper/319:25:143
868899116,3cyclictest19048-21grep21:00:133
868899111,7cyclictest0-21swapper/320:40:023
868899111,6cyclictest0-21swapper/320:01:013
868899110,10cyclictest0-21swapper/321:11:213
868799118,2cyclictest8526-21kworker/u8:120:37:042
868799118,2cyclictest0-21swapper/222:45:092
868799118,2cyclictest0-21swapper/219:50:162
868799118,2cyclictest0-21swapper/200:00:142
868799112,5cyclictest0-21swapper/220:02:582
868799111,7cyclictest0-21swapper/220:07:582
868699111,8cyclictest0-21swapper/120:05:161
868699110,9cyclictest0-21swapper/120:55:141
868699110,9cyclictest0-21swapper/120:41:011
868699110,10cyclictest0-21swapper/119:15:311
868599112,6cyclictest0-21swapper/020:04:010
868899102,7cyclictest0-21swapper/320:48:013
868899101,3cyclictest3728-21runrttasks20:51:193
868799100,7cyclictest0-21swapper/220:15:022
868699108,1cyclictest0-21swapper/119:40:151
868699107,2cyclictest0-21swapper/120:45:121
868699107,2cyclictest0-21swapper/119:51:031
116412100,2sleep20-21swapper/220:40:162
86889996,2cyclictest0-21swapper/321:05:163
86889994,3cyclictest208442sleep319:40:223
86889994,3cyclictest17974-21telnet19:35:133
86889994,2cyclictest31280-21telnet20:10:113
86889992,4cyclictest0-21swapper/320:28:023
86879985,2cyclictest0-21swapper/221:04:022
86879984,3cyclictest21589-21aten2.4_r3power19:45:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional