You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-15 - 11:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Feb 15, 2025 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24519992720,6cyclictest21596-21awk20:15:003
2451699271,2cyclictest1667-21cron19:30:010
2451999260,0cyclictest364-21munin-run22:05:003
2451999250,1cyclictest26429-21munin-run20:25:003
2451999250,0cyclictest0-21swapper/319:48:033
24519992418,1cyclictest23259-21apt-get23:47:543
2451899240,23cyclictest8889-21apt-get22:42:592
2451699240,0cyclictest0-21swapper/023:23:100
2451999230,22cyclictest32119-21cron22:00:003
2451999230,22cyclictest19118-21perf21:20:003
2451899230,2cyclictest4581-21apt-get22:23:112
2451899230,22cyclictest593-21dbus-daemon23:15:002
2451899230,22cyclictest2252-21turbostat.cron20:45:002
24517992322,0cyclictest211rcuc/122:45:001
24517992321,2cyclictest22-21ksoftirqd/120:23:021
24517992321,1cyclictest1017-21turbostat22:10:011
2451799230,22cyclictest23913-21grep21:29:591
2451799230,22cyclictest10800-21latency_hist22:50:001
2451799230,22cyclictest0-21swapper/122:23:101
2451699230,22cyclictest29770-21munin-plugin-st21:55:010
2451699230,22cyclictest0-21swapper/023:58:020
24519992220,1cyclictest612-21gdbus00:15:003
24519992220,1cyclictest0-21swapper/323:45:013
2451999220,21cyclictest13025-21awk23:04:593
2451999220,21cyclictest0-21swapper/321:50:013
2451999220,1cyclictest20450-21munin-run23:35:013
24518992221,1cyclictest6650-21apt-get00:28:032
24518992220,1cyclictest14996-21turbostat23:20:002
2451899220,21cyclictest313-21systemd-journal00:20:012
2451899220,21cyclictest24991-21apt-get19:13:022
2451899220,21cyclictest0-21swapper/223:25:012
2451899220,1cyclictest686-21cat00:09:592
2451899220,1cyclictest4086-21wc22:20:002
2451899220,1cyclictest14256-21perf21:10:002
2451899220,1cyclictest14256-21perf21:10:002
2451899220,1cyclictest0-21swapper/220:10:002
2451899220,0cyclictest0-21swapper/222:00:512
24517992220,1cyclictest14412-21perf19:59:591
2451799220,21cyclictest4073-21awk22:20:001
2451799220,21cyclictest16708-21apt21:15:011
2451799220,21cyclictest0-21swapper/121:52:591
2451799220,1cyclictest8409-21sendmail-msp22:40:001
2451799220,1cyclictest0-21swapper/119:40:001
24516992221,1cyclictest21998-21apt-get21:28:010
24516992220,1cyclictest1836-21apt-get00:17:580
24516992220,1cyclictest0-21swapper/000:23:020
2451699220,21cyclictest24572-21munin-plugin-st21:34:590
2451699220,21cyclictest0-21swapper/021:48:010
2451699220,21cyclictest0-21swapper/021:23:010
2451699220,21cyclictest0-21swapper/021:03:020
2451699220,21cyclictest0-21swapper/020:13:030
2451699220,1cyclictest0-21swapper/021:35:330
24519992121,0cyclictest0-21swapper/323:37:593
24519992121,0cyclictest0-21swapper/323:23:093
24519992121,0cyclictest0-21swapper/321:35:343
24519992120,1cyclictest9948-21apt-get21:03:013
24519992120,1cyclictest6497-21perf19:40:003
24519992120,0cyclictest14791-21apt-get21:13:003
2451999211,0cyclictest17888-21apt-get23:27:593
2451999210,20cyclictest0-21swapper/322:13:273
2451999210,0cyclictest31846-21apt-get20:40:023
2451999210,0cyclictest0-21swapper/321:08:013
2451999210,0cyclictest0-21swapper/321:08:013
24518992120,1cyclictest4586-21apt-get19:38:052
2451899210,2cyclictest10792-21cron22:50:002
2451899210,1cyclictest24573-21munin-run21:34:592
2451799210,20cyclictest28255-21idleruntime-cro00:00:001
2451799210,20cyclictest0-21swapper/121:18:061
2451799210,1cyclictest9470-21awk21:00:001
2451799210,1cyclictest31831-21sh20:39:591
2451799210,1cyclictest11580-21sendmail00:40:001
2451799210,0cyclictest0-21swapper/119:28:031
24516992121,0cyclictest0-21swapper/023:37:580
24516992121,0cyclictest0-21swapper/020:53:030
24516992119,1cyclictest690-21munin-run00:09:590
24516992119,1cyclictest19795-21perf23:30:000
24516992114,6cyclictest0-21swapper/020:58:040
2451699210,20cyclictest313-21systemd-journal19:20:010
2451699210,0cyclictest0-21swapper/020:23:010
24519992020,0cyclictest0-21swapper/322:23:103
24519992020,0cyclictest0-21swapper/320:58:043
24519992020,0cyclictest0-21swapper/319:23:033
24519992019,1cyclictest9691-21apt-get00:37:593
24519992019,0cyclictest26313-21apt-get23:58:013
24518992020,0cyclictest28760-21apt-get21:52:592
24518992020,0cyclictest22086-21apt-get20:18:052
24518992020,0cyclictest22086-21apt-get20:18:042
24518992020,0cyclictest0-21swapper/221:13:012
24518992020,0cyclictest0-21swapper/221:03:012
24518992020,0cyclictest0-21swapper/200:03:042
2451899200,0cyclictest0-21swapper/219:28:022
24517992020,0cyclictest0-21swapper/121:38:081
24517992020,0cyclictest0-21swapper/121:34:591
24517992020,0cyclictest0-21swapper/120:43:021
24517992020,0cyclictest0-21swapper/119:23:041
24517992020,0cyclictest0-21swapper/119:18:011
24517992018,1cyclictest19802-21munin-plugin-st23:30:001
24517992018,1cyclictest14358-21dump-pmu-power23:15:001
2451799200,20cyclictest0-21swapper/120:53:011
2451799200,0cyclictest0-21swapper/121:03:011
24516992020,0cyclictest2746-21apt-get20:48:020
24516992020,0cyclictest0-21swapper/021:13:000
24516992020,0cyclictest0-21swapper/000:37:580
24516992017,2cyclictest313-21systemd-journal00:35:000
24519991919,0cyclictest10809-21apt-get22:50:013
24519991919,0cyclictest0-21swapper/320:48:033
24519991919,0cyclictest0-21swapper/320:08:043
24519991919,0cyclictest0-21swapper/300:23:013
24519991919,0cyclictest0-21swapper/300:17:583
24519991919,0cyclictest0-21swapper/300:03:043
2451999190,1cyclictest25837-21munin-plugin-st23:54:593
2451999190,18cyclictest2189-21apt-get19:33:013
24518991919,0cyclictest26906-21apt-get20:28:062
24518991919,0cyclictest20927-21apt-get23:37:582
24518991919,0cyclictest0-21swapper/222:50:422
24518991919,0cyclictest0-21swapper/221:57:562
24518991919,0cyclictest0-21swapper/219:43:052
24518991919,0cyclictest0-21swapper/219:33:022
24518991918,1cyclictest4258-21apt-get00:23:032
24518991918,0cyclictest0-21swapper/223:27:592
24517991919,0cyclictest0-21swapper/123:47:541
24517991919,0cyclictest0-21swapper/123:23:101
24517991919,0cyclictest0-21swapper/121:57:571
24517991919,0cyclictest0-21swapper/121:23:011
24517991919,0cyclictest0-21swapper/119:43:041
24517991919,0cyclictest0-21swapper/119:33:021
24517991918,1cyclictest0-21swapper/119:13:021
24516991919,0cyclictest0-21swapper/020:43:020
24516991919,0cyclictest0-21swapper/020:28:040
24516991919,0cyclictest0-21swapper/020:05:430
24516991919,0cyclictest0-21swapper/019:48:020
24516991919,0cyclictest0-21swapper/019:33:010
24516991919,0cyclictest0-21swapper/000:03:050
24516991917,1cyclictest7766-21munin-run22:35:000
2451699190,19cyclictest0-21swapper/020:37:570
24519991818,0cyclictest31214-21apt-get00:08:023
24519991818,0cyclictest27388-21apt-get19:18:013
2451999180,16cyclictest313-21systemd-journal22:30:003
24518991818,0cyclictest24564-21apt-get21:35:342
24518991818,0cyclictest17195-21apt-get21:18:062
24518991818,0cyclictest0-21swapper/222:13:282
24518991818,0cyclictest0-21swapper/219:18:012
24518991818,0cyclictest0-21swapper/200:37:572
24518991817,1cyclictest26410-21latency_hist20:25:012
24518991816,1cyclictest1-21systemd20:40:002
24518991816,1cyclictest0-21swapper/219:25:002
24517991818,0cyclictest0-21swapper/122:13:281
24517991818,0cyclictest0-21swapper/120:48:021
24517991818,0cyclictest0-21swapper/120:28:041
24516991818,0cyclictest0-21swapper/023:47:550
24516991818,0cyclictest0-21swapper/022:50:430
24516991818,0cyclictest0-21swapper/022:23:100
24516991818,0cyclictest0-21swapper/022:00:520
24516991818,0cyclictest0-21swapper/019:57:570
24516991818,0cyclictest0-21swapper/019:38:040
24516991816,1cyclictest0-21swapper/020:03:070
2451699180,5cyclictest17163-21/usr/sbin/munin21:18:110
2451699180,17cyclictest22753-21cron23:45:010
24519991716,1cyclictest32202-21apt-get19:28:023
24519991715,1cyclictest15020-21munin-run23:19:593
24518991717,0cyclictest0-21swapper/220:48:022
24517991716,1cyclictest0-21swapper/122:50:421
24517991714,2cyclictest7105-21ls22:29:591
24516991717,0cyclictest0-21swapper/020:18:040
24516991717,0cyclictest0-21swapper/020:18:040
24516991716,1cyclictest0-21swapper/021:08:020
24516991716,1cyclictest0-21swapper/021:08:010
24519991616,0cyclictest0-21swapper/319:13:023
24519991616,0cyclictest0-21swapper/300:28:033
2451999160,8cyclictest0-21swapper/321:23:013
2451999160,1cyclictest0-21swapper/320:30:003
2451999160,15cyclictest593-21dbus-daemon22:40:013
2451999160,13cyclictest0-21swapper/321:45:013
24518991616,0cyclictest12421-21apt-get19:57:562
24518991614,1cyclictest0-21swapper/222:10:002
2451899160,15cyclictest14904-21apt-get20:03:062
2451799167,8cyclictest23176-21munin-node23:54:081
24517991616,0cyclictest22-21ksoftirqd/123:37:581
24517991616,0cyclictest0-21swapper/100:03:051
24517991615,1cyclictest0-21swapper/120:13:031
24516991616,0cyclictest0-21swapper/022:45:010
24516991614,1cyclictest29459-21perf20:35:000
24516991614,1cyclictest25806-21cron23:55:000
24516991614,1cyclictest0-21swapper/019:13:050
24519991515,0cyclictest0-21swapper/322:43:003
24519991515,0cyclictest0-21swapper/320:53:023
24519991515,0cyclictest0-21swapper/320:03:063
24519991515,0cyclictest0-21swapper/300:32:073
2451999150,6cyclictest612-21gdbus20:00:003
24518991515,0cyclictest28093-21apt-get21:48:002
24518991515,0cyclictest0-21swapper/223:47:542
24518991515,0cyclictest0-21swapper/220:13:022
24518991513,1cyclictest0-21swapper/222:30:002
2451899150,1cyclictest27215-21memory23:58:062
2451899150,10cyclictest313-21systemd-journal22:35:002
2451899150,0cyclictest5141-21apt-get20:53:022
24517991515,0cyclictest9392-21apt-get19:48:021
24517991515,0cyclictest0-21swapper/121:48:001
24517991514,1cyclictest22-21ksoftirqd/123:15:221
24517991514,1cyclictest22-21ksoftirqd/100:19:221
24517991514,1cyclictest0-21swapper/121:08:011
24517991514,1cyclictest0-21swapper/121:08:011
24517991513,1cyclictest0-21swapper/100:08:051
24516991513,1cyclictest0-21swapper/000:15:010
24519991414,0cyclictest0-21swapper/322:56:083
24519991414,0cyclictest0-21swapper/320:30:263
24519991414,0cyclictest0-21swapper/320:16:323
24519991414,0cyclictest0-21swapper/320:16:323
2451999140,14cyclictest0-21swapper/323:13:143
2451999140,14cyclictest0-21swapper/323:08:573
2451999140,14cyclictest0-21swapper/322:50:083
2451999140,14cyclictest0-21swapper/322:33:283
2451999140,14cyclictest0-21swapper/322:05:403
2451999140,14cyclictest0-21swapper/321:53:533
2451999140,14cyclictest0-21swapper/321:30:283
2451999140,14cyclictest0-21swapper/319:53:083
2451999140,14cyclictest0-21swapper/319:42:543
2451999140,0cyclictest0-21swapper/322:15:533
2451999140,0cyclictest0-21swapper/321:25:173
2451999140,0cyclictest0-21swapper/320:35:393
24518991414,0cyclictest19605-21apt-get21:23:022
24518991414,0cyclictest0-21swapper/219:52:232
2451899140,14cyclictest0-21swapper/223:51:462
2451899140,14cyclictest0-21swapper/223:07:152
2451899140,14cyclictest0-21swapper/223:01:572
2451899140,14cyclictest0-21swapper/222:56:132
2451899140,14cyclictest0-21swapper/222:38:112
2451899140,14cyclictest0-21swapper/221:25:532
2451899140,14cyclictest0-21swapper/220:55:492
2451899140,14cyclictest0-21swapper/200:12:242
2451899140,0cyclictest0-21swapper/221:41:122
2451899140,0cyclictest0-21swapper/220:30:142
2451899140,0cyclictest0-21swapper/200:30:552
24517991414,0cyclictest0-21swapper/123:42:071
24517991414,0cyclictest0-21swapper/123:30:041
24517991414,0cyclictest0-21swapper/122:57:371
24517991414,0cyclictest0-21swapper/122:31:111
24517991414,0cyclictest0-21swapper/100:30:511
24517991414,0cyclictest0-21swapper/100:26:241
2451799140,14cyclictest0-21swapper/123:00:081
2451799140,14cyclictest0-21swapper/121:41:271
2451799140,14cyclictest0-21swapper/120:31:101
2451799140,14cyclictest0-21swapper/120:00:031
2451799140,14cyclictest0-21swapper/119:50:471
2451799140,0cyclictest0-21swapper/122:03:501
2451799140,0cyclictest0-21swapper/120:15:221
2451799140,0cyclictest0-21swapper/120:15:221
2451799140,0cyclictest0-21swapper/100:10:401
24516991414,0cyclictest0-21swapper/022:56:420
24516991414,0cyclictest0-21swapper/022:13:190
24516991414,0cyclictest0-21swapper/021:55:110
24516991414,0cyclictest0-21swapper/021:41:360
24516991414,0cyclictest0-21swapper/019:24:580
2451699140,14cyclictest0-21swapper/023:15:160
2451699140,14cyclictest0-21swapper/023:12:230
2451699140,14cyclictest0-21swapper/023:07:350
2451699140,14cyclictest0-21swapper/023:01:470
2451699140,14cyclictest0-21swapper/022:46:040
2451699140,14cyclictest0-21swapper/022:15:410
2451699140,14cyclictest0-21swapper/000:27:240
2451699140,0cyclictest0-21swapper/023:31:380
2451699140,0cyclictest0-21swapper/022:36:020
2451699140,0cyclictest0-21swapper/022:26:330
2451699140,0cyclictest0-21swapper/022:06:290
2451699140,0cyclictest0-21swapper/019:51:310
2451699140,0cyclictest0-21swapper/019:42:120
24518991313,0cyclictest0-21swapper/223:40:432
24518991313,0cyclictest0-21swapper/223:30:072
24518991313,0cyclictest0-21swapper/219:49:532
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional