You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 18:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Fri Nov 21, 2025 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14537992422,1cyclictest24159-21cron07:30:003
14532992422,1cyclictest8677-21dpkg12:29:591
1453299240,23cyclictest17578-21sh09:30:001
14537992321,1cyclictest22953-21apt-get10:47:053
1453799230,22cyclictest30865-21apt-get09:57:043
14536992320,2cyclictest20406-21dump-pmu-power09:35:002
1453699230,22cyclictest5174-21latency_hist09:05:002
1453299230,2cyclictest16603-21perf11:40:001
1453299230,22cyclictest27343-21munin-run10:55:001
1453299230,22cyclictest25251-21turbostat.cron09:44:591
1453199231,1cyclictest0-21swapper/008:44:590
1453199230,22cyclictest20402-21munin-run09:34:590
14537992220,1cyclictest15907-21cron08:20:003
1453799220,21cyclictest23861-21awk11:54:593
1453799220,21cyclictest0-21swapper/312:07:063
1453799220,21cyclictest0-21swapper/308:24:593
1453799220,21cyclictest0-21swapper/307:55:003
14536992220,1cyclictest30833-21/usr/sbin/munin09:57:042
14536992220,1cyclictest24603-21apt-get07:32:062
14536992220,1cyclictest15261-21ls10:30:002
14536992220,1cyclictest15261-21ls10:30:002
1453699220,21cyclictest4298-21apt-get12:22:062
1453699220,21cyclictest2857-21perf10:04:592
1453699220,21cyclictest0-21swapper/212:37:072
1453699220,21cyclictest0-21swapper/210:47:042
1453699220,21cyclictest0-21swapper/207:47:052
1453699220,1cyclictest16341-21apt-get08:22:062
1453699220,1cyclictest10797-21apt-get09:17:062
14532992220,1cyclictest7952-21cron09:10:011
1453299220,21cyclictest9814-21apt-get11:27:041
1453299220,21cyclictest3206-21apt-get09:02:061
1453299220,21cyclictest31890-21apt-get12:12:061
1453299220,21cyclictest24166-21cron07:30:011
1453299220,21cyclictest17334-21apt-get07:17:041
1453299220,21cyclictest15212-21munin-run09:25:001
1453299220,21cyclictest0-21swapper/111:35:011
1453299220,21cyclictest0-21swapper/111:02:071
1453299220,21cyclictest0-21swapper/108:47:071
1453299220,21cyclictest0-21swapper/107:50:001
1453299220,21cyclictest0-21swapper/107:42:061
1453299220,0cyclictest0-21swapper/111:17:141
14531992220,1cyclictest30169-21apt-get11:02:070
14531992220,1cyclictest15687-21apt-get10:32:030
14531992220,1cyclictest0-21swapper/008:47:070
1453199220,21cyclictest19015-21latency_hist11:45:000
1453199220,21cyclictest0-21swapper/008:07:060
1453199220,19cyclictest13-21ksoftirqd/012:12:050
14537992121,0cyclictest11889-21apt-get12:37:083
14537992121,0cyclictest0-21swapper/308:47:073
14537992120,1cyclictest0-21swapper/312:32:063
14537992120,0cyclictest0-21swapper/312:22:063
14537992119,1cyclictest27320-21apt10:55:003
14537992119,1cyclictest0-21swapper/310:14:593
1453799210,21cyclictest0-21swapper/310:42:053
1453799210,20cyclictest32562-21apt-get11:07:063
1453799210,20cyclictest1882-21apt-get12:17:073
1453799210,20cyclictest0-21swapper/311:57:063
1453799210,20cyclictest0-21swapper/311:42:043
1453799210,20cyclictest0-21swapper/311:02:073
1453799210,20cyclictest0-21swapper/310:32:043
1453799210,20cyclictest0-21swapper/309:07:053
1453799210,20cyclictest0-21swapper/308:27:043
1453799210,1cyclictest13212-21apt-get09:22:053
1453799210,0cyclictest30798-21apt-get08:52:063
1453799210,0cyclictest30798-21apt-get08:52:063
14536992121,0cyclictest4300-21apt-get07:57:032
14536992120,1cyclictest10371-21perf09:15:002
1453699210,20cyclictest28403-21apt-get08:47:072
1453699210,20cyclictest0-21swapper/211:07:052
1453699210,0cyclictest0-21swapper/211:02:072
14532992121,0cyclictest0-21swapper/109:12:061
14532992119,1cyclictest3888-21awk12:20:011
14532992119,1cyclictest13878-21perf12:40:001
14532992119,1cyclictest0-21swapper/108:10:001
1453299210,21cyclictest0-21swapper/110:02:041
1453299210,21cyclictest0-21swapper/109:45:021
1453299210,0cyclictest27747-21apt-get10:57:051
1453299210,0cyclictest0-21swapper/107:22:061
14531992121,0cyclictest0-21swapper/008:17:030
14531992121,0cyclictest0-21swapper/007:52:030
14531992120,0cyclictest0-21swapper/011:37:060
14531992119,1cyclictest5801-21apt-get09:05:020
14531992118,2cyclictest593-21dbus-daemon07:40:000
1453199210,21cyclictest0-21swapper/009:27:050
1453199210,20cyclictest0-21swapper/012:17:060
1453199210,20cyclictest0-21swapper/011:27:050
1453199210,20cyclictest0-21swapper/007:27:030
1453199210,0cyclictest0-21swapper/009:42:030
1453199210,0cyclictest0-21swapper/008:22:060
14537992020,0cyclictest25248-21apt-get09:45:023
14537992020,0cyclictest0-21swapper/309:32:053
14537992020,0cyclictest0-21swapper/307:37:063
14537992020,0cyclictest0-21swapper/307:12:043
14537992019,1cyclictest3272-21apt-get10:07:063
14537992018,1cyclictest6491-21sed11:17:143
1453799200,20cyclictest0-21swapper/309:52:043
1453799200,0cyclictest0-21swapper/308:37:023
14536992020,0cyclictest9116-21apt-get08:07:052
14536992020,0cyclictest28466-21apt-get09:52:032
14536992020,0cyclictest24283-21apt-get11:57:062
14536992020,0cyclictest23584-21apt-get08:37:022
14536992020,0cyclictest0-21swapper/212:17:072
14536992020,0cyclictest0-21swapper/211:52:072
14536992020,0cyclictest0-21swapper/210:20:002
14536992020,0cyclictest0-21swapper/210:07:052
14536992020,0cyclictest0-21swapper/207:52:032
14536992019,1cyclictest0-21swapper/207:45:002
14536992018,1cyclictest8684-21idleruntime-cro12:30:002
14536992018,1cyclictest12213-21apt-get11:32:062
14536992016,3cyclictest313-21systemd-journal10:15:002
1453699200,20cyclictest28688-21apt-get12:05:002
1453699200,0cyclictest0-21swapper/208:42:032
1453699200,0cyclictest0-21swapper/208:02:042
14532992020,0cyclictest6718-21apt-get08:02:041
14532992020,0cyclictest0-21swapper/112:07:061
14532992020,0cyclictest0-21swapper/112:02:051
14532992020,0cyclictest0-21swapper/110:32:041
14532992020,0cyclictest0-21swapper/107:12:041
14532992019,1cyclictest27104-21apt-get07:37:061
14532992019,1cyclictest13081-21apt-get10:25:021
14532992019,1cyclictest13081-21apt-get10:25:021
14532992019,1cyclictest0-21swapper/110:07:061
14532992018,1cyclictest18987-21cron11:45:001
14532992018,1cyclictest0-21swapper/109:20:001
1453299200,20cyclictest0-21swapper/108:12:081
1453299200,19cyclictest0-21swapper/110:17:041
1453299200,0cyclictest20501-21apt-get10:42:031
1453299200,0cyclictest0-21swapper/111:57:051
1453299200,0cyclictest0-21swapper/111:50:001
14531992020,0cyclictest6693-21apt-get12:27:040
14531992020,0cyclictest13278-21apt-get10:27:050
14531992020,0cyclictest13278-21apt-get10:27:040
14531992020,0cyclictest0-21swapper/010:17:040
14531992020,0cyclictest0-21swapper/009:57:050
14531992020,0cyclictest0-21swapper/009:22:050
14531992020,0cyclictest0-21swapper/009:02:060
14531992020,0cyclictest0-21swapper/008:37:020
14531992020,0cyclictest0-21swapper/008:32:060
14531992020,0cyclictest0-21swapper/007:57:020
14531992019,1cyclictest0-21swapper/007:22:060
14531992017,0cyclictest0-21swapper/010:02:040
1453199200,0cyclictest0-21swapper/010:07:050
1453199200,0cyclictest0-21swapper/009:52:040
14537991919,0cyclictest0-21swapper/312:27:043
14537991919,0cyclictest0-21swapper/310:57:043
14537991919,0cyclictest0-21swapper/308:02:033
14537991919,0cyclictest0-21swapper/307:32:063
1453799190,0cyclictest20813-21apt-get09:37:093
14536991919,0cyclictest22203-21apt-get07:27:032
14536991919,0cyclictest17037-21apt-get11:42:032
14536991919,0cyclictest10485-21apt-get10:22:042
14536991919,0cyclictest0-21swapper/210:32:042
14536991918,1cyclictest18736-21apt-get08:27:032
14536991917,1cyclictest16929-21awk07:14:592
14536991917,1cyclictest0-21swapper/208:55:002
14536991917,1cyclictest0-21swapper/208:55:002
14532991919,0cyclictest0-21swapper/112:32:071
14532991919,0cyclictest0-21swapper/111:22:061
14532991919,0cyclictest0-21swapper/111:07:051
14532991919,0cyclictest0-21swapper/110:47:051
14532991919,0cyclictest0-21swapper/108:27:041
14532991919,0cyclictest0-21swapper/107:57:021
14532991917,1cyclictest21887-21apt-get11:52:071
14531991919,0cyclictest0-21swapper/012:37:080
14531991919,0cyclictest0-21swapper/011:17:060
14531991919,0cyclictest0-21swapper/011:13:130
14531991919,0cyclictest0-21swapper/010:47:050
14531991919,0cyclictest0-21swapper/010:22:050
14531991919,0cyclictest0-21swapper/009:12:060
14531991919,0cyclictest0-21swapper/008:02:040
14531991919,0cyclictest0-21swapper/007:42:050
14531991918,1cyclictest0-21swapper/011:57:060
14531991918,0cyclictest0-21swapper/011:07:050
14531991918,0cyclictest0-21swapper/007:32:060
1453199190,0cyclictest18086-21apt-get10:37:070
14537991818,0cyclictest0-21swapper/311:27:063
14537991818,0cyclictest0-21swapper/310:27:053
14537991818,0cyclictest0-21swapper/310:27:053
14537991818,0cyclictest0-21swapper/310:22:043
14537991817,0cyclictest0-21swapper/311:12:073
14537991816,1cyclictest0-21swapper/310:05:003
14536991818,0cyclictest5997-21apt-get09:07:042
14536991818,0cyclictest0-21swapper/211:27:062
14536991818,0cyclictest0-21swapper/209:45:022
14536991817,1cyclictest0-21swapper/210:37:072
1453699180,17cyclictest0-21swapper/208:57:052
14532991818,0cyclictest26003-21apt-get08:42:031
14532991818,0cyclictest21136-21apt-get08:32:061
14532991818,0cyclictest0-21swapper/110:37:061
14532991818,0cyclictest0-21swapper/109:57:051
14532991818,0cyclictest0-21swapper/108:37:021
14532991818,0cyclictest0-21swapper/107:52:021
14531991818,0cyclictest19435-21apt-get11:47:050
14531991818,0cyclictest0-21swapper/012:32:060
14531991818,0cyclictest0-21swapper/012:07:050
14531991818,0cyclictest0-21swapper/011:52:070
14531991818,0cyclictest0-21swapper/009:47:040
14531991817,1cyclictest13-21ksoftirqd/010:57:040
14531991817,0cyclictest0-21swapper/010:12:070
14531991816,1cyclictest13477-21idleruntime-cro08:15:010
14537991717,0cyclictest0-21swapper/311:37:063
14537991717,0cyclictest0-21swapper/311:32:073
14537991717,0cyclictest0-21swapper/311:22:073
14537991717,0cyclictest0-21swapper/310:37:073
14537991717,0cyclictest0-21swapper/309:27:063
14537991717,0cyclictest0-21swapper/309:02:063
14536991717,0cyclictest7418-21apt-get11:22:072
14536991717,0cyclictest28688-21apt-get12:05:022
14536991717,0cyclictest19756-21apt-get07:22:072
14536991717,0cyclictest0-21swapper/211:17:062
14536991717,0cyclictest0-21swapper/208:32:062
14536991716,1cyclictest593-21dbus-daemon08:20:012
14536991716,0cyclictest23278-21apt-get09:42:032
14532991717,0cyclictest0-21swapper/108:17:031
14532991717,0cyclictest0-21swapper/107:32:051
14531991717,0cyclictest0-21swapper/010:42:040
14531991714,2cyclictest31916-21apt-get07:47:050
14537991616,0cyclictest0-21swapper/308:32:063
14537991614,1cyclictest0-21swapper/312:15:003
14536991616,0cyclictest0-21swapper/209:37:092
14536991616,0cyclictest0-21swapper/209:22:052
14536991616,0cyclictest0-21swapper/207:37:062
14536991614,1cyclictest0-21swapper/211:47:072
1453699160,15cyclictest10010-21tune2fs12:32:082
14532991616,0cyclictest0-21swapper/110:22:041
1453299160,15cyclictest0-21swapper/109:52:041
14531991616,0cyclictest0-21swapper/012:02:030
14531991616,0cyclictest0-21swapper/011:32:070
1453199160,0cyclictest0-21swapper/008:27:040
14537991515,0cyclictest0-21swapper/311:47:043
14537991515,0cyclictest0-21swapper/308:57:053
14537991515,0cyclictest0-21swapper/308:12:083
1453799150,1cyclictest0-21swapper/309:14:593
1453799150,1cyclictest0-21swapper/307:17:103
1453799150,14cyclictest0-21swapper/310:18:203
1453799150,14cyclictest0-21swapper/307:57:043
14536991515,0cyclictest0-21swapper/208:12:072
14536991514,1cyclictest1686-21apache210:51:142
14536991513,1cyclictest32329-21fschecks_count12:12:082
14536991513,1cyclictest16620-21grep11:39:592
1453299150,1cyclictest0-21swapper/108:52:131
1453299150,1cyclictest0-21swapper/108:52:131
1453299150,14cyclictest0-21swapper/109:37:151
14531991515,0cyclictest0-21swapper/008:52:060
14531991515,0cyclictest0-21swapper/008:52:060
14531991513,1cyclictest12774-21cron09:20:000
1453199150,1cyclictest0-21swapper/007:17:100
14537991414,0cyclictest0-21swapper/309:17:053
14537991414,0cyclictest0-21swapper/307:47:053
1453799140,14cyclictest0-21swapper/312:01:343
1453799140,14cyclictest0-21swapper/309:41:173
1453799140,14cyclictest0-21swapper/308:05:373
1453799140,14cyclictest0-21swapper/307:21:583
1453799140,0cyclictest0-21swapper/308:40:003
1453799140,0cyclictest0-21swapper/307:40:203
14536991414,0cyclictest0-21swapper/210:57:052
14536991414,0cyclictest0-21swapper/209:25:332
1453699140,14cyclictest0-21swapper/210:40:062
1453699140,14cyclictest0-21swapper/207:15:382
1453699140,0cyclictest0-21swapper/211:10:532
14532991414,0cyclictest0-21swapper/112:20:101
1453299140,14cyclictest0-21swapper/109:34:591
1453299140,14cyclictest0-21swapper/108:56:311
1453299140,0cyclictest0-21swapper/110:11:561
14531991414,0cyclictest0-21swapper/011:20:580
14531991414,0cyclictest0-21swapper/010:53:070
14531991414,0cyclictest0-21swapper/007:12:050
1453199140,14cyclictest0-21swapper/012:20:380
1453199140,14cyclictest0-21swapper/009:35:330
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional