You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-12 - 09:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack3slot6s.osadl.org (updated Mon Jan 12, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12655211996,18sleep30-21swapper/319:09:033
12596211897,16sleep10-21swapper/119:08:171
12716211496,12sleep20-21swapper/219:09:502
12630210796,6sleep00-21swapper/019:08:440
301062820,0sleep00-21swapper/022:22:570
170152730,1sleep00-21swapper/020:40:130
103122700,0sleep30-21swapper/320:20:253
284312680,2sleep31275999cyclictest21:45:133
39722540,1sleep10-21swapper/123:41:391
163362430,1sleep316344-21unixbench_singl20:35:273
1275899371,18cyclictest0-21swapper/222:25:132
198502290,0sleep30-21swapper/323:24:003
1275899221,2cyclictest0-21swapper/222:30:122
1275899221,19cyclictest11648-21ssh23:15:122
1275899221,19cyclictest0-21swapper/223:25:132
1275899221,19cyclictest0-21swapper/220:50:102
169122210,0sleep20-21swapper/222:08:122
1275699211,3cyclictest0-21swapper/020:20:100
1275699210,2cyclictest0-21swapper/020:45:110
1275699210,2cyclictest0-21swapper/019:30:110
12758992015,3cyclictest24181-21ssh00:05:112
1275699200,18cyclictest0-21swapper/019:40:100
1275899190,14cyclictest0-21swapper/223:46:042
1275999180,16cyclictest28838-21diskmemload22:25:133
1275999180,13cyclictest0-21swapper/323:43:033
1275999180,13cyclictest0-21swapper/323:12:053
1275999180,13cyclictest0-21swapper/300:35:053
12757991815,2cyclictest29055-21kworker/u8:122:51:051
12757991815,2cyclictest16984-21kworker/u8:021:03:051
1275799180,3cyclictest0-21swapper/100:03:041
1275999170,15cyclictest0-21swapper/321:25:103
1275999170,12cyclictest0-21swapper/320:05:053
12758991712,3cyclictest17270-21kworker/u8:023:50:052
12757991714,2cyclictest6276-21kworker/u8:020:24:051
12757991714,2cyclictest16984-21kworker/u8:020:54:041
12757991714,2cyclictest14275-21kworker/u8:200:36:051
12757991714,2cyclictest14101-21kworker/u8:123:21:041
12757991713,3cyclictest0-21swapper/121:25:101
1275699171,14cyclictest9787-21fschecks_count22:00:150
1275999160,14cyclictest0-21swapper/300:11:243
1275999160,12cyclictest0-21swapper/322:36:053
1275999160,11cyclictest0-21swapper/323:25:053
1275899162,2cyclictest0-21swapper/222:10:052
1275799161,3cyclictest7458-21runrttasks23:50:161
12757991613,2cyclictest6276-21kworker/u8:020:17:041
12757991613,2cyclictest4615-21kworker/u8:021:58:041
12757991613,2cyclictest32368-21kworker/u8:000:15:051
12757991613,2cyclictest30572-21kworker/u8:222:06:041
12757991613,2cyclictest29208-21kworker/u8:220:12:051
12757991613,2cyclictest25156-21kworker/u8:019:48:051
12757991613,2cyclictest25156-21kworker/u8:019:43:051
12757991613,2cyclictest22550-21kworker/u8:219:34:051
12757991613,2cyclictest20111-21kworker/u8:223:26:051
12757991613,2cyclictest17270-21kworker/u8:023:35:041
12757991613,2cyclictest16984-21kworker/u8:020:55:051
12757991613,2cyclictest13019-21kworker/u8:120:36:051
12757991613,2cyclictest10944-21kworker/u8:219:10:041
12757991613,2cyclictest10389-21kworker/u8:220:32:051
12757991613,2cyclictest10389-21kworker/u8:220:29:041
1275799161,2cyclictest0-21swapper/121:46:051
1275799161,13cyclictest6050-21taskset22:31:561
1275799161,13cyclictest0-21swapper/123:48:471
1275799161,13cyclictest0-21swapper/121:10:121
1275799160,14cyclictest0-21swapper/121:31:481
1275799160,14cyclictest0-21swapper/121:31:481
1275799160,13cyclictest26172-21taskset22:55:061
12756991612,2cyclictest16984-21kworker/u8:021:12:050
12756991611,3cyclictest193052chrt21:35:120
12756991611,3cyclictest193052chrt21:35:110
91032150,0sleep00-21swapper/022:00:000
12759991511,2cyclictest5114-21latency_hist00:20:013
12759991510,3cyclictest26832-21munin-plugin-st21:04:593
1275999150,13cyclictest15233-21ssh23:55:073
1275899152,7cyclictest0-21swapper/219:39:592
1275899151,3cyclictest13209-21ssh22:40:122
12758991512,2cyclictest4137-21kworker/u8:223:06:042
1275899150,3cyclictest26543-21switchtime23:30:252
1275899150,13cyclictest11118-21switchtime00:25:262
12757991512,2cyclictest4137-21kworker/u8:223:16:051
12757991512,2cyclictest29208-21kworker/u8:220:01:041
12757991512,2cyclictest25156-21kworker/u8:019:51:041
12757991512,2cyclictest22550-21kworker/u8:219:35:041
12757991512,2cyclictest17270-21kworker/u8:023:00:051
12757991512,2cyclictest14275-21kworker/u8:200:32:051
12757991512,2cyclictest14275-21kworker/u8:200:32:041
12757991512,2cyclictest13773-21kworker/u8:223:59:031
12757991510,3cyclictest0-21swapper/100:25:181
1275799150,2cyclictest0-21swapper/122:13:051
1275799150,2cyclictest0-21swapper/121:16:041
1275799150,13cyclictest0-21swapper/123:14:041
1275799150,13cyclictest0-21swapper/122:39:391
1275799150,13cyclictest0-21swapper/119:17:051
1275699150,13cyclictest0-21swapper/022:35:140
93922140,2sleep20-21swapper/221:23:402
166262140,2sleep00-21swapper/022:07:440
1275999141,2cyclictest6675-21snmpd21:16:273
12759991411,2cyclictest28838-21diskmemload23:02:513
12759991411,2cyclictest0-21swapper/322:55:093
12759991411,2cyclictest0-21swapper/300:20:413
1275999140,2cyclictest0-21swapper/321:10:163
1275999140,12cyclictest4544-21ssh23:06:013
1275999140,12cyclictest27146-21apt-get22:20:113
1275999140,12cyclictest19691-21sed22:10:263
1275999140,12cyclictest0-21swapper/323:50:283
1275999140,12cyclictest0-21swapper/321:30:333
1275999140,12cyclictest0-21swapper/321:30:323
1275999140,12cyclictest0-21swapper/300:34:413
1275999140,12cyclictest0-21swapper/300:34:403
1275899142,7cyclictest0-21swapper/221:30:012
1275899142,7cyclictest0-21swapper/221:30:012
1275899142,6cyclictest0-21swapper/221:00:002
12758991411,2cyclictest17270-21kworker/u8:022:56:042
12758991411,2cyclictest14101-21kworker/u8:123:24:042
1275899141,11cyclictest1714-21munin-node2
1275799149,3cyclictest0-21swapper/122:01:261
12757991411,2cyclictest28838-21diskmemload22:26:291
12757991411,2cyclictest0-21swapper/121:21:181
1275799140,2cyclictest0-21swapper/123:05:391
1275799140,13cyclictest0-21swapper/122:45:081
1275799140,12cyclictest28838-21diskmemload00:10:251
1275799140,12cyclictest0-21swapper/123:34:041
1275799140,12cyclictest0-21swapper/122:15:181
1275799140,12cyclictest0-21swapper/121:40:041
1275699149,3cyclictest17523-21grep19:20:160
1275699149,3cyclictest0-21swapper/022:45:130
1275699140,2cyclictest0-21swapper/023:11:540
1275699140,12cyclictest0-21swapper/023:50:070
1275699140,12cyclictest0-21swapper/022:15:150
36962130,2sleep00-21swapper/020:05:250
1275999139,2cyclictest0-21swapper/319:35:123
12759991310,2cyclictest28838-21diskmemload22:47:513
12759991310,2cyclictest0-21swapper/323:46:213
12759991310,2cyclictest0-21swapper/323:37:003
12759991310,2cyclictest0-21swapper/323:15:053
12759991310,2cyclictest0-21swapper/322:42:373
12759991310,2cyclictest0-21swapper/322:00:213
12759991310,2cyclictest0-21swapper/321:40:183
12759991310,2cyclictest0-21swapper/321:36:393
12759991310,2cyclictest0-21swapper/321:36:393
12759991310,2cyclictest0-21swapper/321:20:473
12759991310,2cyclictest0-21swapper/320:10:233
12759991310,2cyclictest0-21swapper/300:05:203
1275999130,2cyclictest0-21swapper/323:30:433
1275999130,2cyclictest0-21swapper/322:34:363
1275999130,2cyclictest0-21swapper/322:15:293
1275999130,2cyclictest0-21swapper/320:25:133
1275999130,2cyclictest0-21swapper/300:25:023
1275999130,12cyclictest0-21swapper/322:05:033
1275999130,12cyclictest0-21swapper/321:53:043
1275999130,12cyclictest0-21swapper/319:25:193
1275999130,12cyclictest0-21swapper/300:00:213
1275899138,3cyclictest8847-21grep20:20:002
1275899138,3cyclictest0-21swapper/222:49:212
12758991310,2cyclictest29208-21kworker/u8:220:11:042
1275899130,11cyclictest1147-21rm23:39:372
12757991310,2cyclictest30572-21kworker/u8:221:50:051
12757991310,2cyclictest0-21swapper/121:38:491
12757991310,2cyclictest0-21swapper/121:38:481
1275799130,2cyclictest328750irq/24-eth122:23:081
1275799130,2cyclictest0-21swapper/119:55:261
1275799130,2cyclictest0-21swapper/119:25:041
1275799130,2cyclictest0-21swapper/100:21:271
1275799130,12cyclictest0-21swapper/122:40:181
1275799130,12cyclictest0-21swapper/100:05:091
1275699139,3cyclictest2122-21ssh00:15:250
1275699131,10cyclictest32099-21perl22:25:160
12756991310,2cyclictest28838-21diskmemload23:30:450
12756991310,2cyclictest28838-21diskmemload00:24:050
12756991310,2cyclictest0-21swapper/023:45:170
12756991310,2cyclictest0-21swapper/023:39:440
12756991310,2cyclictest0-21swapper/021:50:100
12756991310,2cyclictest0-21swapper/000:05:050
1275699130,2cyclictest17267-21ssh00:32:510
1275699130,2cyclictest17267-21ssh00:32:500
1275699130,2cyclictest0-21swapper/022:57:300
1275699130,2cyclictest0-21swapper/000:10:270
1275699130,12cyclictest0-21swapper/021:44:450
1275699130,12cyclictest0-21swapper/021:16:040
1275699130,11cyclictest0-21swapper/022:11:410
1275699130,11cyclictest0-21swapper/000:37:080
233332120,2sleep00-21swapper/022:50:370
175532120,3sleep317617-21if_eth020:40:183
13802120,2sleep20-21swapper/223:03:252
1275999129,2cyclictest0-21swapper/319:30:183
1275999129,2cyclictest0-21swapper/319:20:243
1275999129,2cyclictest0-21swapper/319:15:333
1275999122,6cyclictest0-21swapper/319:52:593
1275999120,11cyclictest16984-21kworker/u8:020:53:043
1275999120,11cyclictest0-21swapper/322:50:313
1275999120,11cyclictest0-21swapper/321:55:183
1275999120,11cyclictest0-21swapper/320:15:253
1275999120,11cyclictest0-21swapper/320:00:233
1275899129,2cyclictest9799-21kworker/u8:121:29:042
1275899129,2cyclictest30572-21kworker/u8:221:47:042
1275899129,2cyclictest29208-21kworker/u8:220:08:052
1275899129,2cyclictest28838-21diskmemload22:52:532
1275899129,2cyclictest25156-21kworker/u8:019:54:052
1275899129,2cyclictest0-21swapper/222:20:252
1275899129,2cyclictest0-21swapper/221:35:252
1275899129,2cyclictest0-21swapper/221:35:242
1275899129,2cyclictest0-21swapper/200:00:132
1275899122,7cyclictest0-21swapper/200:20:582
1275899121,6cyclictest0-21swapper/221:55:012
1275899121,6cyclictest0-21swapper/221:45:002
1275899120,8cyclictest0-21swapper/219:16:052
1275899120,2cyclictest0-21swapper/220:45:262
1275899120,11cyclictest0-21swapper/200:17:022
1275899120,10cyclictest0-21swapper/200:32:292
1275899120,10cyclictest0-21swapper/200:32:292
1275899120,10cyclictest0-21swapper/200:10:582
1275799129,2cyclictest0-21swapper/120:45:011
1275799120,2cyclictest0-21swapper/121:05:261
1275799120,2cyclictest0-21swapper/120:41:041
1275799120,11cyclictest0-21swapper/120:05:121
1275699129,2cyclictest21092-21aten2.4-expect20:50:110
1275699129,2cyclictest0-21swapper/023:25:150
1275699129,2cyclictest0-21swapper/023:19:110
1275699129,2cyclictest0-21swapper/023:03:450
1275699129,2cyclictest0-21swapper/022:31:250
1275699129,2cyclictest0-21swapper/019:34:590
1275699120,2cyclictest0-21swapper/023:08:410
1275699120,2cyclictest0-21swapper/022:43:450
1275699120,2cyclictest0-21swapper/021:32:580
1275699120,2cyclictest0-21swapper/021:32:570
1275699120,2cyclictest0-21swapper/021:20:150
1275699120,2cyclictest0-21swapper/000:01:230
1275699120,11cyclictest0-21swapper/023:22:440
1275699120,11cyclictest0-21swapper/021:46:350
1275699120,11cyclictest0-21swapper/021:25:210
1275699120,11cyclictest0-21swapper/000:27:170
1275699120,10cyclictest0-21swapper/023:40:330
1275999118,2cyclictest0-21swapper/320:30:273
1275999110,10cyclictest25156-21kworker/u8:019:55:043
1275999110,10cyclictest21131-21kworker/u8:220:56:053
1275899118,2cyclictest29208-21kworker/u8:220:04:042
1275899118,2cyclictest0-21swapper/222:35:272
1275899113,6cyclictest0-21swapper/219:56:112
1275899112,6cyclictest0-21swapper/220:56:592
1275899111,7cyclictest0-21swapper/220:41:592
1275899111,6cyclictest0-21swapper/223:55:232
1275899110,9cyclictest20811-21munin-run19:30:002
1275899110,7cyclictest0-21swapper/220:29:592
1275899110,2cyclictest28838-21diskmemload22:17:012
1275899110,2cyclictest0-21swapper/223:10:412
1275899110,10cyclictest0-21swapper/221:11:152
1275899110,10cyclictest0-21swapper/200:35:292
1275799110,3cyclictest0-21swapper/119:20:151
1275699118,2cyclictest0-21swapper/023:56:550
1275699118,2cyclictest0-21swapper/021:00:180
1275699113,6cyclictest0-21swapper/020:16:130
1275699110,10cyclictest0-21swapper/020:56:460
1275699110,10cyclictest0-21swapper/019:45:200
120792110,2sleep20-21swapper/222:02:412
56712100,1sleep00-21swapper/020:10:220
166172100,2sleep20-21swapper/220:38:232
1275899107,2cyclictest27144-21kworker/u8:221:06:052
1275899107,2cyclictest25156-21kworker/u8:019:45:042
1275899107,2cyclictest13019-21kworker/u8:120:31:052
1275899105,3cyclictest5053-21ssh21:19:162
1275899102,6cyclictest0-21swapper/220:21:022
1275899101,7cyclictest25322-21grep19:40:122
1275699105,3cyclictest13917-21proc_pri19:10:180
1275699102,6cyclictest0-21swapper/021:06:240
1275699102,6cyclictest0-21swapper/020:35:260
1275699102,6cyclictest0-21swapper/020:26:150
1275699102,6cyclictest0-21swapper/019:16:030
1275699101,7cyclictest7458-21runrttasks20:03:230
1275699101,5cyclictest0-21swapper/020:35:000
127599994,3cyclictest26355-21sed19:40:243
127599990,8cyclictest27144-21kworker/u8:221:08:053
127599990,8cyclictest0-21swapper/319:12:513
127589995,2cyclictest29-21rcuc/219:10:122
127589994,3cyclictest22651-21seq19:32:592
127589994,3cyclictest17564-21cut19:20:152
127589991,3cyclictest6217-21grep21:55:242
127569996,2cyclictest0-21swapper/019:55:120
127569994,3cyclictest19281-21grep19:25:120
127599984,2cyclictest27160-21cut19:45:103
127599983,3cyclictest6318-21Xorg20:45:593
127569984,2cyclictest29589-21tune2fs19:50:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional