You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 18:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Mon Dec 01, 2025 12:44:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1240199270,14cyclictest0-21swapper/309:27:583
12394992523,1cyclictest27753-21apt-get07:42:591
1240199249,8cyclictest27807-21cron09:54:593
12401992422,1cyclictest34-21ksoftirqd/307:30:003
1240199240,23cyclictest0-21swapper/310:40:023
12396992422,1cyclictest593-21dbus-daemon07:14:592
1239499240,23cyclictest14729-21cron07:15:001
1240199230,22cyclictest26536-21latency_hist08:45:003
1240199230,1cyclictest0-21swapper/310:10:003
1239699230,16cyclictest13418-21tr11:37:592
12394992321,1cyclictest768-21runrttasks10:45:001
1239499230,1cyclictest313-21systemd-journal07:25:001
1239499230,0cyclictest0-21swapper/109:42:571
1239099231,21cyclictest0-21swapper/012:05:000
1239099230,22cyclictest6832-21dump-pmu-power12:30:010
1239099230,22cyclictest0-21swapper/012:35:010
12401992221,1cyclictest18039-21apt-get07:22:563
1240199220,21cyclictest25266-21apt-get12:02:593
1240199220,21cyclictest0-21swapper/311:52:573
1240199220,21cyclictest0-21swapper/309:15:013
1240199220,21cyclictest0-21swapper/309:02:583
1240199220,21cyclictest0-21swapper/308:28:003
1240199220,1cyclictest8275-21apt10:20:003
1240199220,1cyclictest28940-21perf08:50:003
1240199220,1cyclictest13080-21ls10:30:003
1240199220,0cyclictest0-21swapper/311:42:583
12396992222,0cyclictest21110-21apt-get09:42:572
1239699220,21cyclictest20593-21cron09:39:592
1239699220,21cyclictest14544-21munin-run08:20:012
1239699220,21cyclictest10832-21apt-get11:32:592
1239699220,1cyclictest17149-21apt-get07:20:022
12394992221,1cyclictest31820-21apt-get08:57:581
12394992221,1cyclictest31820-21apt-get08:57:581
12394992220,1cyclictest5852-21turbostat.cron10:15:001
12394992220,1cyclictest10110-21apt-get12:37:581
1239499220,21cyclictest6197-21perf09:10:011
1239499220,1cyclictest27235-21idleruntime-cro07:40:001
1239499220,1cyclictest20597-21apt09:40:001
12390992220,1cyclictest23092-21latency_hist10:50:010
1239099220,21cyclictest0-21swapper/012:32:570
1239099220,21cyclictest0-21swapper/011:22:580
1239099220,21cyclictest0-21swapper/010:17:580
1239099220,21cyclictest0-21swapper/009:42:570
1239099220,21cyclictest0-21swapper/009:02:580
1239099220,21cyclictest0-21swapper/007:47:570
1239099220,21cyclictest0-21swapper/007:17:580
1239099220,21cyclictest0-21swapper/007:17:580
1239099220,21cyclictest0-21swapper/007:12:580
1239099220,1cyclictest17908-21sendmail-msp10:40:000
1239099220,1cyclictest0-21swapper/012:09:590
12401992121,0cyclictest0-21swapper/309:42:573
12401992121,0cyclictest0-21swapper/307:17:593
12401992121,0cyclictest0-21swapper/307:17:583
12401992119,1cyclictest24754-21turbostat07:35:013
12401992118,2cyclictest20635-21perf09:40:003
1240199210,21cyclictest7562-21perf11:25:003
1240199210,20cyclictest12824-21apt-get07:12:573
1240199210,20cyclictest0-21swapper/308:12:573
1240199210,1cyclictest0-21swapper/311:30:023
1240199210,0cyclictest23570-21apt-get10:52:583
1240199210,0cyclictest0-21swapper/308:52:583
1240199210,0cyclictest0-21swapper/308:02:583
12396992121,0cyclictest0-21swapper/208:38:022
12396992121,0cyclictest0-21swapper/207:42:582
12396992120,1cyclictest22849-21apt-get11:57:592
1239699210,20cyclictest19811-21apt-get08:32:562
1239699210,20cyclictest0-21swapper/211:13:032
1239699210,20cyclictest0-21swapper/210:02:572
1239699210,0cyclictest0-21swapper/207:37:592
12394992121,0cyclictest0-21swapper/111:52:571
12394992121,0cyclictest0-21swapper/107:52:571
12394992120,0cyclictest0-21swapper/110:02:571
1239499210,21cyclictest0-21swapper/110:52:581
1239499210,21cyclictest0-21swapper/108:38:021
1239499210,20cyclictest0-21swapper/111:57:591
1239499210,20cyclictest0-21swapper/108:42:571
1239499210,1cyclictest6677-21apt-get09:12:571
1239499210,0cyclictest749-21apt-get11:13:031
1239499210,0cyclictest12603-21apt-get08:17:591
1239499210,0cyclictest0-21swapper/112:02:591
1239499210,0cyclictest0-21swapper/107:27:571
12390992121,0cyclictest0-21swapper/011:07:590
12390992120,1cyclictest30231-21apt-get10:00:020
12390992120,1cyclictest0-21swapper/010:22:580
12390992120,0cyclictest0-21swapper/010:42:580
1239099210,20cyclictest0-21swapper/012:12:590
1239099210,20cyclictest0-21swapper/011:57:590
1239099210,1cyclictest29422-21apt-get08:52:590
1239099210,0cyclictest0-21swapper/009:27:580
1239099210,0cyclictest0-21swapper/009:22:580
1239099210,0cyclictest0-21swapper/009:12:570
1239099210,0cyclictest0-21swapper/008:47:580
12401992020,0cyclictest0-21swapper/311:37:583
12401992020,0cyclictest0-21swapper/310:47:583
12401992020,0cyclictest0-21swapper/307:57:573
12401992020,0cyclictest0-21swapper/307:55:003
12401992019,1cyclictest0-21swapper/312:35:013
12401992019,0cyclictest15968-21apt-get10:38:013
12401992018,1cyclictest9211-21cron12:35:003
12401992018,1cyclictest5873-21munin-run10:15:003
12401992018,1cyclictest14526-21turbostat08:20:013
12401992017,2cyclictest34-21ksoftirqd/310:00:013
1240199200,20cyclictest0-21swapper/308:22:583
1240199200,0cyclictest8038-21apt-get11:28:003
1240199200,0cyclictest0-21swapper/312:07:573
1240199200,0cyclictest0-21swapper/309:08:013
12396992020,0cyclictest5640-21apt-get11:22:582
12396992020,0cyclictest20446-21apt-get11:52:572
12396992020,0cyclictest11160-21apt-get10:27:572
12396992020,0cyclictest0-21swapper/212:37:592
12396992020,0cyclictest0-21swapper/210:52:582
12396992020,0cyclictest0-21swapper/208:42:572
12396992019,1cyclictest593-21dbus-daemon08:49:592
12396992019,1cyclictest20684-21perf10:45:002
1239699200,20cyclictest0-21swapper/212:32:562
1239699200,20cyclictest0-21swapper/209:12:572
1239699200,0cyclictest0-21swapper/211:07:592
1239699200,0cyclictest0-21swapper/210:12:582
12394992020,0cyclictest9080-21apt-get09:18:001
12394992020,0cyclictest25969-21apt-get10:58:001
12394992020,0cyclictest0-21swapper/108:27:581
12394992019,0cyclictest13235-21apt-get11:37:581
12394992019,0cyclictest0-21swapper/111:47:591
12394992018,1cyclictest2109-21apt-get07:55:021
12394992018,1cyclictest18212-21perf09:35:001
1239499200,20cyclictest0-21swapper/111:22:571
1239499200,20cyclictest0-21swapper/110:27:571
1239499200,20cyclictest0-21swapper/109:57:581
1239499200,0cyclictest0-21swapper/111:28:001
12390992020,0cyclictest1090-21apt-get10:07:580
12390992020,0cyclictest10832-21apt-get11:32:590
12390992020,0cyclictest0-21swapper/010:57:590
12390992020,0cyclictest0-21swapper/010:27:580
12390992020,0cyclictest0-21swapper/010:12:580
12390992020,0cyclictest0-21swapper/009:18:010
12390992020,0cyclictest0-21swapper/008:07:570
12390992020,0cyclictest0-21swapper/008:07:560
12390992020,0cyclictest0-21swapper/007:57:590
12390992020,0cyclictest0-21swapper/007:52:570
12390992020,0cyclictest0-21swapper/007:42:590
12390992019,1cyclictest0-21swapper/010:52:580
1239099200,0cyclictest0-21swapper/011:52:570
1239099200,0cyclictest0-21swapper/009:52:580
12401991919,0cyclictest0-21swapper/312:23:023
12401991917,1cyclictest0-21swapper/309:00:003
12401991917,1cyclictest0-21swapper/308:59:593
12396991919,0cyclictest28302-21apt-get09:57:582
12396991919,0cyclictest0-21swapper/207:52:572
12396991918,1cyclictest0-21swapper/208:55:002
12396991917,1cyclictest3084-21munin-run10:10:002
1239699190,19cyclictest0-21swapper/208:57:572
1239699190,19cyclictest0-21swapper/208:57:572
12394991919,0cyclictest3083-21dpkg10:10:001
12394991919,0cyclictest13898-21apt-get09:27:571
12394991919,0cyclictest0-21swapper/112:32:571
12394991919,0cyclictest0-21swapper/111:03:001
12394991918,0cyclictest0-21swapper/110:32:571
12394991917,1cyclictest27826-21cat09:55:001
12394991917,1cyclictest0-21swapper/110:39:591
12390991919,0cyclictest0-21swapper/008:38:020
12390991919,0cyclictest0-21swapper/008:28:000
12390991919,0cyclictest0-21swapper/008:17:590
12390991919,0cyclictest0-21swapper/008:12:570
12390991918,1cyclictest0-21swapper/011:13:030
1239099190,1cyclictest1989-21cron12:20:000
12401991818,0cyclictest8764-21apt-get10:22:583
12401991818,0cyclictest0-21swapper/309:32:573
12396991818,0cyclictest17419-21apt-get08:27:592
12396991818,0cyclictest0-21swapper/210:58:002
12396991818,0cyclictest0-21swapper/209:18:012
12396991817,1cyclictest0-21swapper/210:32:582
12394991818,0cyclictest9940-21apt-get11:30:021
12394991818,0cyclictest0-21swapper/108:32:551
12394991818,0cyclictest0-21swapper/107:32:561
1239499180,17cyclictest8279-21munin-run10:20:001
12390991818,0cyclictest0-21swapper/012:23:020
12390991818,0cyclictest0-21swapper/011:17:590
1239099180,0cyclictest0-21swapper/011:03:000
12401991717,0cyclictest28388-21apt-get11:03:003
12401991717,0cyclictest0-21swapper/308:32:553
12401991716,1cyclictest25337-21apt-get07:38:003
12401991716,1cyclictest0-21swapper/308:38:023
12401991715,1cyclictest25969-21apt-get10:57:583
12396991717,0cyclictest16293-21apt-get09:32:562
12396991717,0cyclictest0-21swapper/211:47:592
12396991717,0cyclictest0-21swapper/211:03:002
12396991717,0cyclictest0-21swapper/209:27:572
1239699170,2cyclictest6176-21cron09:10:012
1239699170,17cyclictest0-21swapper/207:27:562
12394991717,0cyclictest5400-21apt-get08:02:581
12394991717,0cyclictest2527-21apt-get12:23:011
12394991717,0cyclictest0-21swapper/111:07:591
12394991717,0cyclictest0-21swapper/107:47:571
12390991717,0cyclictest13564-21apt-get10:32:580
12390991717,0cyclictest0-21swapper/009:57:570
12390991717,0cyclictest0-21swapper/008:02:580
12390991717,0cyclictest0-21swapper/007:32:570
12401991616,0cyclictest0-21swapper/312:13:003
12401991616,0cyclictest0-21swapper/311:13:023
12401991614,1cyclictest0-21swapper/312:27:353
12401991614,1cyclictest0-21swapper/311:18:053
12396991616,0cyclictest30148-21apt-get07:47:562
12396991616,0cyclictest0-21swapper/211:27:592
12396991616,0cyclictest0-21swapper/207:55:022
12396991616,0cyclictest0-21swapper/207:32:572
1239699160,1cyclictest1884-21apt-get09:02:582
12394991616,0cyclictest0-21swapper/109:02:581
12394991616,0cyclictest0-21swapper/108:12:571
12394991614,1cyclictest0-21swapper/110:25:001
12394991614,1cyclictest0-21swapper/107:18:081
12394991614,1cyclictest0-21swapper/107:18:081
12390991616,0cyclictest0-21swapper/007:20:020
12390991614,2cyclictest20592-21readlink09:40:000
1239099160,15cyclictest15820-21irqstats08:23:010
12401991515,0cyclictest31113-21apt-get10:02:573
12401991515,0cyclictest0-21swapper/309:22:583
12401991513,1cyclictest9707-21apt08:10:003
12401991513,1cyclictest9707-21apt08:10:003
12396991515,0cyclictest10192-21apt-get08:12:572
12396991515,0cyclictest0-21swapper/212:23:022
12396991515,0cyclictest0-21swapper/208:02:582
12396991514,1cyclictest17892-21cat10:40:002
12396991513,1cyclictest26262-21strings12:03:042
12396991513,1cyclictest0-21swapper/209:53:012
12394991515,0cyclictest0-21swapper/111:17:591
1239099156,8cyclictest0-21swapper/008:33:040
12390991515,0cyclictest23509-21apt-get09:47:580
12390991515,0cyclictest0-21swapper/008:57:570
12390991515,0cyclictest0-21swapper/008:57:570
12390991515,0cyclictest0-21swapper/007:27:560
12390991513,1cyclictest17593-21cut11:45:000
1239099150,1cyclictest9960-21munin-run11:30:000
12401991414,0cyclictest0-21swapper/309:17:423
1240199140,14cyclictest0-21swapper/311:47:443
1240199140,14cyclictest0-21swapper/307:46:123
1240199140,14cyclictest0-21swapper/307:42:033
1240199140,0cyclictest0-21swapper/312:15:043
1240199140,0cyclictest0-21swapper/311:57:363
1240199140,0cyclictest0-21swapper/311:07:463
1240199140,0cyclictest0-21swapper/310:31:533
12396991412,2cyclictest15163-21munin-node11:16:522
1239699140,14cyclictest0-21swapper/212:28:042
1239699140,14cyclictest0-21swapper/212:16:312
1239699140,14cyclictest0-21swapper/212:12:092
1239699140,14cyclictest0-21swapper/211:41:152
1239699140,14cyclictest0-21swapper/209:46:512
1239699140,14cyclictest0-21swapper/209:20:032
1239699140,14cyclictest0-21swapper/208:22:122
1239699140,14cyclictest0-21swapper/207:16:462
1239699140,14cyclictest0-21swapper/207:16:462
1239699140,0cyclictest0-21swapper/212:05:462
1239699140,0cyclictest0-21swapper/210:20:312
1239699140,0cyclictest0-21swapper/210:15:172
12394991414,0cyclictest0-21swapper/112:15:161
12394991414,0cyclictest0-21swapper/110:45:441
12394991414,0cyclictest0-21swapper/109:47:291
1239499140,14cyclictest0-21swapper/112:25:361
1239499140,14cyclictest0-21swapper/111:42:201
1239499140,14cyclictest0-21swapper/108:46:231
1239499140,14cyclictest0-21swapper/108:24:231
1239499140,0cyclictest0-21swapper/112:11:431
1239499140,0cyclictest0-21swapper/108:51:081
1239499140,0cyclictest0-21swapper/108:07:121
1239499140,0cyclictest0-21swapper/108:07:121
12390991414,0cyclictest0-21swapper/011:35:330
12390991414,0cyclictest0-21swapper/009:32:300
1239099140,14cyclictest0-21swapper/011:46:490
1239099140,14cyclictest0-21swapper/007:35:080
12396991313,0cyclictest0-21swapper/208:07:122
12396991313,0cyclictest0-21swapper/208:07:122
12390991313,0cyclictest0-21swapper/008:42:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional