You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-07 - 08:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Fri Aug 07, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28779992727,0cyclictest34-21ksoftirqd/300:11:193
2877999260,24cyclictest34-21ksoftirqd/322:12:183
28779992524,1cyclictest34-21ksoftirqd/322:04:413
28779992523,1cyclictest5284-21apt-get21:24:093
2877999251,24cyclictest34-21ksoftirqd/321:32:183
2877999250,24cyclictest0-21swapper/300:08:463
2877599250,24cyclictest22821-21diskmemload22:08:462
2877599250,24cyclictest22821-21diskmemload00:13:422
28771992523,1cyclictest26420-21apt-get21:14:051
2877199250,24cyclictest0-21swapper/121:35:281
2877199250,1cyclictest28526-21ssh22:13:531
28769992523,1cyclictest0-21swapper/022:02:300
2876999250,24cyclictest0-21swapper/023:32:330
2876999250,24cyclictest0-21swapper/000:06:300
2876999250,1cyclictest0-21swapper/023:55:210
28779992424,0cyclictest0-21swapper/300:02:233
28779992423,1cyclictest22821-21diskmemload23:51:553
2877999240,24cyclictest0-21swapper/322:52:173
2877999240,23cyclictest22821-21diskmemload23:15:313
2877999240,23cyclictest22821-21diskmemload22:06:523
2877999240,23cyclictest0-21swapper/322:37:533
2877999240,23cyclictest0-21swapper/321:12:293
2877999240,0cyclictest0-21swapper/323:04:073
28775992424,0cyclictest0-21swapper/200:21:392
28775992421,2cyclictest26319-21sh21:14:002
2877599240,23cyclictest6803-21perf23:20:002
2877599240,23cyclictest3366-21sed22:19:142
2877599240,23cyclictest2874-21munin-run23:45:002
2877599240,23cyclictest22821-21diskmemload21:16:252
2877599240,23cyclictest14771-21ssh23:27:562
28771992424,0cyclictest22821-21diskmemload23:53:011
28771992423,1cyclictest10298-21ssh00:20:541
2877199240,3cyclictest1097-21awk23:15:001
2877199240,24cyclictest22821-21diskmemload21:50:051
2877199240,1cyclictest0-21swapper/100:35:171
28769992423,0cyclictest22821-21diskmemload21:53:190
28769992422,1cyclictest0-21swapper/000:17:330
2876999240,2cyclictest4308-21awk22:20:000
2876999240,24cyclictest0-21swapper/023:06:240
2876999240,24cyclictest0-21swapper/021:32:020
2876999240,23cyclictest0-21swapper/022:09:050
2876999240,23cyclictest0-21swapper/000:14:220
28779992323,0cyclictest22821-21diskmemload21:16:033
28779992323,0cyclictest0-21swapper/323:57:433
28779992323,0cyclictest0-21swapper/322:30:413
28779992323,0cyclictest0-21swapper/300:27:143
28779992322,1cyclictest34-21ksoftirqd/323:14:053
28779992322,1cyclictest34-21ksoftirqd/322:57:203
28779992321,1cyclictest22821-21diskmemload22:49:063
28779992321,1cyclictest0-21swapper/323:25:203
28779992320,2cyclictest3585-21latency_hist20:30:013
2877999231,21cyclictest34-21ksoftirqd/322:25:053
2877999230,23cyclictest34-21ksoftirqd/321:46:283
2877999230,23cyclictest34-21ksoftirqd/321:46:283
2877999230,22cyclictest4140-21sh23:46:323
2877999230,22cyclictest23641-21rm23:06:213
2877999230,22cyclictest22821-21diskmemload22:21:363
2877999230,22cyclictest10129-21ssh00:20:373
2877999230,22cyclictest0-21swapper/322:19:553
2877999230,1cyclictest22821-21diskmemload21:39:053
2877999230,0cyclictest0-21swapper/323:38:013
2877999230,0cyclictest0-21swapper/321:51:423
2877999230,0cyclictest0-21swapper/300:32:293
2877999230,0cyclictest0-21swapper/300:15:023
28775992323,0cyclictest22821-21diskmemload21:26:512
28775992323,0cyclictest0-21swapper/223:37:282
28775992323,0cyclictest0-21swapper/222:46:002
28775992323,0cyclictest0-21swapper/222:30:242
28775992323,0cyclictest0-21swapper/221:52:402
28775992323,0cyclictest0-21swapper/221:42:552
28775992322,1cyclictest22821-21diskmemload21:24:222
28775992321,1cyclictest23109-21ssh22:36:572
28775992320,2cyclictest28415-21ssh22:13:452
2877599230,2cyclictest10566-21rm22:25:432
2877599230,23cyclictest0-21swapper/222:55:142
2877599230,23cyclictest0-21swapper/200:35:492
2877599230,23cyclictest0-21swapper/200:07:522
2877599230,22cyclictest30620-21ssh23:13:272
2877599230,22cyclictest25138-21awk00:05:002
2877599230,22cyclictest14273-21ssh21:31:122
2877599230,1cyclictest0-21swapper/223:01:382
2877599230,0cyclictest0-21swapper/223:20:072
2877599230,0cyclictest0-21swapper/223:20:072
2877599230,0cyclictest0-21swapper/222:40:512
2877599230,0cyclictest0-21swapper/200:26:222
28771992323,0cyclictest22821-21diskmemload23:05:401
28771992323,0cyclictest0-21swapper/123:45:581
28771992323,0cyclictest0-21swapper/100:32:411
28771992322,1cyclictest22821-21diskmemload22:54:161
28771992321,1cyclictest11169-21ssh22:26:291
28771992320,2cyclictest807-21ssh22:45:371
2877199230,2cyclictest2680-21ssh00:14:111
2877199230,23cyclictest6471-21ssh22:22:431
2877199230,23cyclictest0-21swapper/123:18:541
2877199230,23cyclictest0-21swapper/122:58:001
2877199230,23cyclictest0-21swapper/122:32:031
2877199230,22cyclictest5394-21ssh00:16:481
2877199230,22cyclictest30848-21ssh21:45:481
2877199230,22cyclictest30848-21ssh21:45:481
2877199230,22cyclictest25530-21ssh00:05:131
2877199230,22cyclictest16547-21ssh23:58:151
2877199230,1cyclictest13244-21munin-run21:30:001
2877199230,0cyclictest22821-21diskmemload00:25:051
2877199230,0cyclictest0-21swapper/122:41:191
2877199230,0cyclictest0-21swapper/121:56:191
2877199230,0cyclictest0-21swapper/121:15:261
28769992323,0cyclictest22821-21diskmemload23:27:420
28769992323,0cyclictest0-21swapper/023:16:420
28769992323,0cyclictest0-21swapper/022:31:400
28769992323,0cyclictest0-21swapper/021:55:330
28769992322,1cyclictest22821-21diskmemload22:50:510
28769992322,0cyclictest0-21swapper/022:24:060
28769992321,1cyclictest62550irq/127-eno100:03:490
28769992321,1cyclictest0-21swapper/023:48:190
28769992321,1cyclictest0-21swapper/022:39:120
28769992320,2cyclictest17451-21ssh23:00:240
2876999230,23cyclictest0-21swapper/022:41:140
2876999230,23cyclictest0-21swapper/021:12:550
2876999230,22cyclictest6346-21idleruntime-cro19:30:010
2876999230,22cyclictest27557-21sh22:12:280
2876999230,22cyclictest26104-21ssh21:42:050
2876999230,22cyclictest19180-21sh21:35:050
2876999230,22cyclictest1055-21sh22:45:560
2876999230,22cyclictest0-21swapper/023:37:410
2876999230,22cyclictest0-21swapper/023:21:290
2876999230,22cyclictest0-21swapper/023:21:290
2876999230,22cyclictest0-21swapper/022:57:380
2876999230,22cyclictest0-21swapper/000:22:490
2876999230,21cyclictest0-21swapper/021:48:200
2876999230,21cyclictest0-21swapper/021:48:200
2876999230,0cyclictest0-21swapper/023:42:520
28779992222,0cyclictest34-21ksoftirqd/322:41:383
28779992222,0cyclictest0-21swapper/323:44:253
28779992222,0cyclictest0-21swapper/323:20:453
28779992222,0cyclictest0-21swapper/323:20:453
28779992220,1cyclictest0-21swapper/300:35:303
2877999220,21cyclictest0-21swapper/323:30:403
2877999220,1cyclictest18040-21turbostat21:00:003
2877999220,1cyclictest12990-21ssh21:29:453
2877999220,1cyclictest0-21swapper/321:43:323
2877999220,0cyclictest0-21swapper/319:19:063
28775992222,0cyclictest0-21swapper/223:47:172
28775992222,0cyclictest0-21swapper/223:33:122
28775992222,0cyclictest0-21swapper/222:52:272
28775992221,1cyclictest17477-21apt-get22:04:042
28775992220,1cyclictest9929-21sh21:56:272
28775992220,1cyclictest21858-21ssh21:38:582
28775992220,1cyclictest0-21swapper/223:59:082
28775992220,1cyclictest0-21swapper/219:35:012
2877599220,21cyclictest8665-21ssh23:50:062
2877599220,21cyclictest4783-21ssh00:16:012
2877599220,21cyclictest0-21swapper/220:45:002
28771992222,0cyclictest0-21swapper/122:37:531
28771992222,0cyclictest0-21swapper/121:30:121
28771992221,1cyclictest13928-21sh23:26:461
28771992221,1cyclictest0-21swapper/121:44:081
2877199220,22cyclictest10046-21apt-get23:24:071
2877199220,22cyclictest10046-21apt-get23:24:071
2877199220,21cyclictest31647-21ssh22:15:231
2877199220,21cyclictest19939-21ssh00:00:011
2877199220,21cyclictest0-21swapper/121:09:061
2877199220,1cyclictest22821-21diskmemload22:02:511
2877199220,0cyclictest22821-21diskmemload22:05:531
28769992218,2cyclictest313-21systemd-journal19:55:000
2876999220,22cyclictest0-21swapper/023:10:210
2876999220,21cyclictest156332sleep000:25:350
2876999220,21cyclictest0-21swapper/022:25:090
2876999220,21cyclictest0-21swapper/021:25:480
2876999220,21cyclictest0-21swapper/021:24:090
2876999220,21cyclictest0-21swapper/020:34:050
2876999220,21cyclictest0-21swapper/019:49:040
2876999220,1cyclictest11630-21apt-get23:54:070
2876999220,1cyclictest0-21swapper/021:16:280
2876999220,1cyclictest0-21swapper/000:32:250
2876999220,0cyclictest0-21swapper/000:36:410
28779992120,1cyclictest606-21dbus-daemon20:40:003
2877999210,0cyclictest0-21swapper/320:24:043
2877999210,0cyclictest0-21swapper/319:59:063
28775992119,1cyclictest6317-21cron19:30:012
28775992119,1cyclictest22834-21grep21:10:012
2877599210,21cyclictest0-21swapper/222:20:392
2877599210,21cyclictest0-21swapper/221:46:062
2877599210,21cyclictest0-21swapper/221:46:062
2877599210,21cyclictest0-21swapper/200:30:142
2877599210,20cyclictest29317-21apt-get19:14:052
2877599210,20cyclictest18625-21apt-get21:04:052
2877599210,0cyclictest26558-21apt-get20:14:042
28771992121,0cyclictest0-21swapper/120:54:031
28771992121,0cyclictest0-21swapper/120:34:051
2877199210,21cyclictest0-21swapper/123:00:051
2877199210,21cyclictest0-21swapper/121:21:331
28769992121,0cyclictest0-21swapper/020:59:050
28769992121,0cyclictest0-21swapper/019:19:060
28769992119,1cyclictest606-21dbus-daemon20:10:000
2877999204,11cyclictest34-21ksoftirqd/320:44:033
28779992018,1cyclictest23587-21sh20:05:003
28779992017,2cyclictest13182-21dump-pmu-power20:50:003
2877999200,20cyclictest0-21swapper/319:44:053
2877999200,20cyclictest0-21swapper/319:14:053
28775992020,0cyclictest28963-21apt-get20:19:042
28775992019,1cyclictest4158-21apt-get20:34:052
28775992019,1cyclictest1751-21apt-get20:29:032
2877599200,20cyclictest0-21swapper/219:54:052
2877599200,0cyclictest0-21swapper/220:09:062
2877599200,0cyclictest0-21swapper/219:19:062
28771992020,0cyclictest0-21swapper/119:14:041
28771992019,1cyclictest32112-21apt-get19:19:061
28771992018,1cyclictest0-21swapper/120:20:011
2877199200,0cyclictest11364-21apt-get20:49:061
28769992020,0cyclictest0-21swapper/020:24:040
2876999200,0cyclictest0-21swapper/020:19:050
2876999200,0cyclictest0-21swapper/020:14:040
2876999200,0cyclictest0-21swapper/019:24:060
28779991919,0cyclictest0-21swapper/319:24:063
28775991919,0cyclictest13526-21apt-get19:45:012
28775991919,0cyclictest0-21swapper/220:39:072
28775991919,0cyclictest0-21swapper/220:24:042
28771991919,0cyclictest0-21swapper/120:24:041
28771991919,0cyclictest0-21swapper/120:04:061
28771991917,1cyclictest11147-21perf19:40:011
28769991919,0cyclictest13188-21perf20:50:000
28769991919,0cyclictest0-21swapper/020:44:030
28769991919,0cyclictest0-21swapper/020:29:040
28779991818,0cyclictest0-21swapper/319:49:043
28779991817,1cyclictest0-21swapper/319:39:053
28779991817,1cyclictest0-21swapper/319:34:053
28775991818,0cyclictest0-21swapper/219:44:052
28771991818,0cyclictest2118-21apt-get19:24:061
28771991818,0cyclictest0-21swapper/121:04:051
28771991818,0cyclictest0-21swapper/119:49:041
28771991815,2cyclictest21202-21munin-plugin-st20:00:001
28769991817,0cyclictest6554-21apt-get20:39:070
28779991717,0cyclictest0-21swapper/321:04:063
2877999170,16cyclictest30796-21perf20:20:013
28775991717,0cyclictest0-21swapper/219:24:062
28771991717,0cyclictest0-21swapper/119:34:051
2877999160,15cyclictest22865-21latency_hist21:10:013
2877999160,15cyclictest0-21swapper/320:34:103
28771991614,1cyclictest4510-21apt-get19:29:061
2877199160,5cyclictest0-21swapper/120:14:041
2877199160,1cyclictest0-21swapper/120:59:051
2876999161,13cyclictest611-21irqbalance21:00:140
28779991515,0cyclictest0-21swapper/320:54:033
28779991515,0cyclictest0-21swapper/319:54:053
28779991515,0cyclictest0-21swapper/319:29:053
28775991515,0cyclictest21772-21apt-get20:04:062
2877199150,1cyclictest3083-21readlink20:29:121
28769991515,0cyclictest0-21swapper/021:09:060
28769991512,2cyclictest606-21dbus-daemon20:00:000
28779991414,0cyclictest0-21swapper/320:05:053
2877999140,14cyclictest0-21swapper/320:10:373
2877599141,12cyclictest13758-21apt-get20:54:032
2877599140,14cyclictest0-21swapper/220:45:432
2877599140,10cyclictest11123-21dump-pmu-power19:40:012
2877599140,0cyclictest0-21swapper/220:56:282
28771991414,0cyclictest0-21swapper/120:35:491
28771991414,0cyclictest0-21swapper/119:41:321
2877199140,0cyclictest0-21swapper/120:40:241
2877199140,0cyclictest0-21swapper/120:06:341
28769991414,0cyclictest0-21swapper/019:40:260
28769991414,0cyclictest0-21swapper/019:10:140
2876999140,14cyclictest0-21swapper/020:50:410
2876999140,14cyclictest0-21swapper/020:01:430
2876999140,14cyclictest0-21swapper/019:31:130
2876999140,0cyclictest0-21swapper/019:38:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional