You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-10 - 16:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon Nov 10, 2025 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3206999280,2cyclictest1580-21cron11:50:003
3206999260,0cyclictest0-21swapper/310:01:083
3206999240,1cyclictest22894-21grep12:31:183
3206999240,1cyclictest0-21swapper/307:16:193
32069992321,1cyclictest0-21swapper/309:35:003
32069992319,3cyclictest0-21swapper/309:51:063
32069992315,7cyclictest4837-21apt-get07:21:063
3206999230,2cyclictest12680-21idleruntime-cro10:00:003
3206999230,22cyclictest999-21idleruntime-cro08:20:013
3206999230,22cyclictest0-21swapper/309:41:103
3206499232,1cyclictest0-21swapper/211:30:012
32064992321,1cyclictest5883-21munin-run08:30:002
32064992321,1cyclictest0-21swapper/209:00:012
3206499230,22cyclictest2028-21perf07:15:002
3206499230,22cyclictest2028-21perf07:14:592
3206499230,22cyclictest15825-21apt-get08:51:092
3206199230,22cyclictest12703-21basename10:00:001
3206199230,22cyclictest0-21swapper/109:00:021
3205999230,22cyclictest28616-21cat08:10:010
3205999230,19cyclictest0-21swapper/012:21:140
32069992219,3cyclictest0-21swapper/308:21:083
3206999220,2cyclictest11272-21grep07:31:163
3206999220,21cyclictest17510-21dump-pmu-power10:09:593
3206999220,21cyclictest0-21swapper/311:56:063
3206999220,21cyclictest0-21swapper/308:56:093
3206999220,1cyclictest1984-21ntp_kernel_err09:36:133
3206999220,1cyclictest16946-21apt-get07:46:083
3206999220,1cyclictest0-21swapper/311:26:173
3206999220,1cyclictest0-21swapper/311:00:003
3206999220,1cyclictest0-21swapper/308:06:173
3206499220,21cyclictest9202-21apt-get12:06:072
3206499220,21cyclictest6451-21latency_hist12:00:002
3206499220,21cyclictest26859-21apt-get09:16:032
3206499220,21cyclictest19194-21munin-run11:20:002
3206499220,1cyclictest583-21avahi-daemon09:50:012
32061992222,0cyclictest0-21swapper/110:46:081
32061992221,1cyclictest31910-21/usr/sbin/munin11:46:101
32061992221,1cyclictest3055-21perf09:40:001
32061992220,1cyclictest6784-21munin-run10:54:591
32061992220,1cyclictest23255-21basename12:35:001
32061992220,1cyclictest16370-21cron11:15:001
3206199220,21cyclictest5864-21perf08:30:001
3206199220,21cyclictest20238-21apt-get10:16:091
3206199220,21cyclictest18618-21apt-get08:56:071
3206199220,21cyclictest15446-21apt-get10:06:091
3206199220,21cyclictest13996-21apt-get12:16:111
3206199220,0cyclictest0-21swapper/108:41:101
32059992220,1cyclictest20688-21apt-get09:00:020
32059992219,2cyclictest13641-21sh12:15:010
3205999220,21cyclictest6445-21tr12:00:000
3205999220,21cyclictest31346-21apt-get08:16:080
3205999220,21cyclictest0-21swapper/012:31:080
3205999220,21cyclictest0-21swapper/012:16:110
3205999220,21cyclictest0-21swapper/011:36:090
3205999220,21cyclictest0-21swapper/011:16:110
3205999220,21cyclictest0-21swapper/011:15:000
3205999220,21cyclictest0-21swapper/010:51:070
3205999220,1cyclictest17896-21perf08:55:000
32069992119,1cyclictest0-21swapper/311:06:083
32069992119,1cyclictest0-21swapper/310:26:093
3206999210,20cyclictest11594-21perf11:05:003
3206999210,20cyclictest0-21swapper/312:36:103
3206999210,1cyclictest11742-21awk08:41:133
3206999210,1cyclictest0-21swapper/312:16:103
3206999210,1cyclictest0-21swapper/307:36:153
3206999210,0cyclictest0-21swapper/307:26:063
32064992119,1cyclictest3062-21awk09:40:002
32064992119,1cyclictest16390-21munin-run11:15:002
32064992119,1cyclictest11600-21apt-get12:11:102
32064992118,2cyclictest23237-21cron12:35:002
3206499210,20cyclictest6215-21apt-get08:31:082
3206499210,20cyclictest3394-21apt-get09:41:102
3206499210,20cyclictest27453-21apt-get10:31:082
3206499210,20cyclictest27123-21apt-get11:36:102
3206499210,20cyclictest10633-21apt-get09:56:072
3206499210,1cyclictest29582-21cron09:25:002
3206499210,1cyclictest21741-21apt-get07:56:102
3206499210,0cyclictest1938-21apt-get11:51:092
3206499210,0cyclictest0-21swapper/210:06:092
32061992121,0cyclictest16946-21apt-get07:46:071
32061992121,0cyclictest0-21swapper/107:36:071
32061992120,0cyclictest0-21swapper/112:26:101
3206199210,21cyclictest0-21swapper/109:16:031
3206199210,20cyclictest29542-21apt-get11:41:081
3206199210,20cyclictest0-21swapper/111:16:111
3206199210,20cyclictest0-21swapper/108:16:081
3206199210,0cyclictest4837-21apt-get07:21:071
3206199210,0cyclictest0-21swapper/109:06:081
32059992121,0cyclictest0-21swapper/011:31:080
32059992121,0cyclictest0-21swapper/009:56:080
32059992121,0cyclictest0-21swapper/009:06:090
32059992120,1cyclictest26859-21apt-get09:16:030
32059992119,1cyclictest0-21swapper/010:29:590
32059992119,1cyclictest0-21swapper/009:15:000
3205999210,21cyclictest1641-21cron12:30:000
3205999210,21cyclictest0-21swapper/009:26:080
3205999210,20cyclictest0-21swapper/010:41:090
3205999210,20cyclictest0-21swapper/010:31:080
3205999210,20cyclictest0-21swapper/010:21:100
3205999210,0cyclictest0-21swapper/011:41:070
3205999210,0cyclictest0-21swapper/007:36:070
32069992020,0cyclictest0-21swapper/311:51:083
32069992020,0cyclictest0-21swapper/311:41:083
32069992020,0cyclictest0-21swapper/311:11:093
32069992020,0cyclictest0-21swapper/310:36:113
32069992020,0cyclictest0-21swapper/310:31:093
32069992020,0cyclictest0-21swapper/309:00:023
32069992020,0cyclictest0-21swapper/308:46:083
32069992019,0cyclictest0-21swapper/307:51:063
32069992016,4cyclictest0-21swapper/308:51:093
3206999200,2cyclictest30061-21phc2sys08:11:173
3206999200,20cyclictest0-21swapper/312:06:083
3206999200,1cyclictest0-21swapper/308:36:093
3206999200,19cyclictest0-21swapper/310:16:123
3206999200,19cyclictest0-21swapper/310:11:123
32064992020,0cyclictest32280-21apt-get10:41:102
32064992020,0cyclictest0-21swapper/211:41:082
32064992020,0cyclictest0-21swapper/210:46:092
32064992020,0cyclictest0-21swapper/208:16:072
32064992020,0cyclictest0-21swapper/207:36:072
32064992019,1cyclictest6808-21apt-get12:01:092
32064992019,0cyclictest7235-21apt-get07:26:072
32064992019,0cyclictest11034-21apt-get08:41:102
3206499200,20cyclictest0-21swapper/211:46:102
3206499200,19cyclictest0-21swapper/208:55:022
32061992020,0cyclictest29854-21apt-get10:36:101
32061992020,0cyclictest21931-21apt-get11:26:091
32061992020,0cyclictest16413-21apt-get12:21:091
32061992020,0cyclictest0-21swapper/112:01:101
32061992020,0cyclictest0-21swapper/110:31:091
32061992019,1cyclictest17844-21apt-get10:11:101
3206199200,1cyclictest26221-21perf08:05:001
3206199200,1cyclictest26221-21perf08:04:591
3206199200,0cyclictest0-21swapper/108:31:081
32059992020,0cyclictest8235-21apt-get09:51:060
32059992020,0cyclictest3766-21apt-get08:26:090
32059992020,0cyclictest0-21swapper/012:36:090
32059992020,0cyclictest0-21swapper/011:26:090
32059992020,0cyclictest0-21swapper/010:16:090
32059992020,0cyclictest0-21swapper/010:11:100
32059992020,0cyclictest0-21swapper/009:46:080
32059992020,0cyclictest0-21swapper/008:55:020
32059992020,0cyclictest0-21swapper/008:11:110
32059992020,0cyclictest0-21swapper/007:56:090
32059992020,0cyclictest0-21swapper/007:31:070
3205999200,20cyclictest0-21swapper/010:01:070
3205999200,20cyclictest0-21swapper/008:01:070
3205999200,20cyclictest0-21swapper/008:01:070
3206999192,16cyclictest14550-21apt-get07:41:083
3206999192,16cyclictest0-21swapper/309:16:073
32069991919,0cyclictest0-21swapper/311:36:103
32069991919,0cyclictest0-21swapper/310:21:103
32069991919,0cyclictest0-21swapper/309:46:083
32069991917,1cyclictest8270-21sh08:34:593
32069991917,1cyclictest24329-21apt-get11:31:083
3206999190,3cyclictest0-21swapper/312:11:183
3206999190,1cyclictest20636-21phc2sys11:21:163
3206999190,1cyclictest0-21swapper/309:06:163
32064991919,0cyclictest7114-21apt-get10:56:072
32064991919,0cyclictest0-21swapper/212:26:102
32064991919,0cyclictest0-21swapper/211:31:082
32064991919,0cyclictest0-21swapper/207:46:072
32061991919,0cyclictest9631-21apt-get07:31:071
32061991917,1cyclictest13428-21apt-get08:46:091
32059991919,0cyclictest0-21swapper/011:01:090
32059991919,0cyclictest0-21swapper/010:56:080
32059991919,0cyclictest0-21swapper/008:46:080
32059991919,0cyclictest0-21swapper/008:36:080
32059991919,0cyclictest0-21swapper/007:26:070
32059991918,1cyclictest4001-21perf11:55:000
32059991918,0cyclictest959-21apt-get09:36:070
3205999190,18cyclictest0-21swapper/009:41:090
32069991818,0cyclictest0-21swapper/311:15:013
32069991818,0cyclictest0-21swapper/308:01:073
32069991818,0cyclictest0-21swapper/308:01:073
3206999181,16cyclictest0-21swapper/312:01:173
3206999180,4cyclictest0-21swapper/309:26:133
3206999180,0cyclictest0-21swapper/312:21:083
32064991818,0cyclictest0-21swapper/212:21:092
32064991818,0cyclictest0-21swapper/210:54:592
32064991818,0cyclictest0-21swapper/210:36:102
32064991817,1cyclictest19540-21apt-get11:21:112
32061991818,0cyclictest5840-21apt-get09:46:071
32061991818,0cyclictest23607-21apt-get12:36:101
32061991818,0cyclictest0-21swapper/112:06:081
32061991818,0cyclictest0-21swapper/111:59:591
32061991818,0cyclictest0-21swapper/111:21:111
32061991818,0cyclictest0-21swapper/108:11:111
32059991818,0cyclictest32376-21apt-get07:11:090
32059991818,0cyclictest32376-21apt-get07:11:090
32059991818,0cyclictest0-21swapper/008:35:000
32059991816,1cyclictest0-21swapper/007:25:010
32069991717,0cyclictest0-21swapper/310:51:073
32069991716,0cyclictest18811-21apt-get12:26:103
32069991713,3cyclictest0-21swapper/310:46:093
32064991717,0cyclictest0-21swapper/212:16:112
32064991717,0cyclictest0-21swapper/210:11:102
32064991717,0cyclictest0-21swapper/207:31:072
32064991716,0cyclictest0-21swapper/210:16:092
3206499170,16cyclictest10291-21sh09:55:012
32061991717,0cyclictest0-21swapper/110:56:081
32061991716,0cyclictest0-21swapper/108:21:081
32061991715,1cyclictest16580-21cron07:45:001
32061991715,1cyclictest13980-21cron11:10:001
3206199170,2cyclictest10286-21sh09:55:001
3206199170,17cyclictest0-21swapper/111:51:071
3205999179,0cyclictest0-21swapper/010:06:080
32059991717,0cyclictest16609-21perf07:45:000
32059991715,1cyclictest4336-21munin-run10:50:000
3206999160,15cyclictest5475-21unixbench_multi08:26:193
3206999160,15cyclictest0-21swapper/310:41:193
32064991616,0cyclictest28954-21apt-get08:11:102
32064991616,0cyclictest0-21swapper/212:36:102
32064991614,1cyclictest12093-21tr11:06:102
32061991616,0cyclictest2365-21apt-get07:16:091
32061991616,0cyclictest0-21swapper/107:56:081
32061991614,1cyclictest27603-21ntp_states08:06:141
32061991614,1cyclictest13226-21grep10:01:081
32059991616,0cyclictest0-21swapper/008:41:090
32059991616,0cyclictest0-21swapper/007:16:090
3205999160,15cyclictest537-21ntp_states11:46:140
3205999160,15cyclictest31943-21awk10:40:000
3206999150,14cyclictest0-21swapper/307:56:143
32064991515,0cyclictest0-21swapper/208:46:092
32064991513,1cyclictest26212-21apt08:05:002
32064991513,1cyclictest26212-21apt08:04:592
32064991513,1cyclictest16594-21basename07:45:002
3206499150,1cyclictest13227-21tr10:01:072
3206499150,1cyclictest0-21swapper/207:51:082
3206499150,14cyclictest0-21swapper/210:21:142
3206499150,12cyclictest781-21lldpd07:19:532
32061991515,0cyclictest0-21swapper/110:21:101
32061991515,0cyclictest0-21swapper/107:11:091
32061991515,0cyclictest0-21swapper/107:11:081
32061991514,0cyclictest24329-21apt-get11:31:081
32061991513,1cyclictest27547-21diskstats11:36:121
32059991515,0cyclictest0-21swapper/011:06:090
32059991515,0cyclictest0-21swapper/007:51:060
32059991513,2cyclictest10522-21mailstats12:06:160
32059991513,1cyclictest0-21swapper/008:21:080
3205999150,14cyclictest0-21swapper/012:01:110
32069991413,1cyclictest0-21swapper/309:13:563
3206999140,14cyclictest0-21swapper/309:20:493
3206999140,0cyclictest0-21swapper/307:10:553
3206999140,0cyclictest0-21swapper/307:10:553
3206499144,6cyclictest1354-21apt-get08:21:072
3206499140,14cyclictest0-21swapper/210:25:572
3206499140,14cyclictest0-21swapper/209:10:102
3206499140,14cyclictest0-21swapper/208:36:302
3206499140,14cyclictest0-21swapper/208:06:022
3206499140,14cyclictest0-21swapper/207:23:432
3206499140,0cyclictest0-21swapper/209:30:422
3206499140,0cyclictest0-21swapper/209:05:202
32061991414,0cyclictest0-21swapper/109:26:171
3206199140,14cyclictest0-21swapper/111:00:451
3206199140,14cyclictest0-21swapper/110:41:161
3206199140,14cyclictest0-21swapper/109:41:171
3206199140,14cyclictest0-21swapper/108:51:111
3206199140,14cyclictest0-21swapper/107:25:131
3206199140,0cyclictest0-21swapper/112:11:181
3206199140,0cyclictest0-21swapper/109:31:101
3206199140,0cyclictest0-21swapper/109:21:131
3206199140,0cyclictest0-21swapper/109:11:011
32059991414,0cyclictest0-21swapper/009:30:010
32059991414,0cyclictest0-21swapper/009:21:580
3205999140,14cyclictest0-21swapper/011:25:000
3205999140,14cyclictest0-21swapper/007:48:180
32064991313,0cyclictest0-21swapper/211:00:442
32064991313,0cyclictest0-21swapper/209:27:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional