You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 14:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Fri Nov 28, 2025 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
312056997433,37cyclictest1-21systemd20:50:002
31205099727,26cyclictest327111-21unin-run00:00:000
312054996827,36cyclictest1-21systemd20:40:011
31205999671,11cyclictest328870-21sshd00:32:353
312059996625,37cyclictest323197-21systemd-run22:45:013
31205999662,16cyclictest313548-21munin-run19:40:013
31205099666,18cyclictest322909-21modprobe22:39:590
312056996521,12cyclictest30-21ksoftirqd/222:40:002
312059996426,33cyclictest320563-21cron21:55:003
31205999642,15cyclictest322660-21munin-run22:35:013
31205999641,14cyclictest321075-21sshd22:01:353
312054996423,37cyclictest322930-21tr22:39:591
31205999632,6cyclictest327600-21munin-run00:10:013
31205999632,6cyclictest327600-21munin-run00:10:013
31205999632,15cyclictest318210-21sshd21:09:343
31205999632,13cyclictest325273-21munin-run23:25:013
312059996317,41cyclictest314809-21cat20:00:023
31205999631,13cyclictest326536-21sshd23:48:183
31205999631,10cyclictest323669-21sshd22:52:343
31205699634,19cyclictest1-21systemd22:35:002
312056996315,13cyclictest1-21systemd00:40:012
312056996315,13cyclictest1-21systemd00:40:002
312056996311,13cyclictest272-21dbus-daemon00:00:012
312054996319,5cyclictest318016-21latency_hist21:05:011
31205099636,21cyclictest328110-21modprobe00:19:590
31205999622,13cyclictest319538-21munin-run21:35:003
31205999622,13cyclictest316939-21munin-run20:45:013
31205999622,13cyclictest316939-21munin-run20:45:003
31205499625,52cyclictest1-21systemd00:20:011
312054996225,32cyclictest313030-21latency_hist19:30:011
312054996224,34cyclictest272-21dbus-daemon23:25:491
31205999613,15cyclictest315351-21modprobe20:15:003
31205999607,7cyclictest324457-21sshd23:08:083
31205999603,18cyclictest326032-21latency_hist23:40:003
31205999602,11cyclictest320289-21sshd21:45:353
31205999601,12cyclictest315091-21sshd20:09:343
31205499601,14cyclictest326195-21grep23:40:011
31205999598,11cyclictest272-21dbus-daemon20:50:013
31205999592,14cyclictest326307-21munin-run23:45:003
31205999592,12cyclictest329133-21sshd00:36:373
31205999592,12cyclictest329133-21sshd00:36:363
31205999591,10cyclictest323417-21sshd22:47:363
31205099591,23cyclictest327793-21sed00:10:030
31205099591,23cyclictest327793-21sed00:10:020
312059995822,32cyclictest325227-21latency_hist23:20:023
31205999582,16cyclictest320837-21munin-run22:00:013
31205999582,14cyclictest322154-21munin-run22:25:003
31205999581,9cyclictest318460-21sshd21:12:353
31205999581,13cyclictest327858-21systemd-run00:15:003
31205699584,12cyclictest328111-21modprobe00:20:002
31205999572,9cyclictest319297-21munin-run21:30:003
31205999572,14cyclictest314042-21sshd19:47:353
31205999571,15cyclictest319013-21sshd21:24:363
31205999571,15cyclictest319013-21sshd21:24:353
31205999571,12cyclictest363-21snmpd21:36:513
31205999571,12cyclictest363-21snmpd21:36:503
312056995719,5cyclictest320751-21latency_hist21:55:032
312056995718,8cyclictest30-21ksoftirqd/223:00:022
312054995725,29cyclictest317411-21sshd20:51:351
312054995719,12cyclictest1-21systemd21:35:011
312054995717,35cyclictest1-21systemd22:45:011
312054995713,10cyclictest314617-21systemctl20:00:011
31205999562,7cyclictest321360-21munin-run22:10:003
31205999562,17cyclictest316686-21sendmail-msp20:40:003
31205999562,15cyclictest325753-21sshd23:33:133
31205999562,11cyclictest363-21snmpd20:50:173
312056995624,7cyclictest326017-21cron23:39:592
312056995619,7cyclictest322254-21cat22:25:022
312054995623,29cyclictest325071-21latency_hist23:20:021
312054995619,32cyclictest324372-21latency_hist23:05:021
312054995615,12cyclictest319289-21systemd-run21:30:001
312054995610,41cyclictest314869-21unin-run20:05:001
31205999552,7cyclictest328154-21munin-run00:20:003
31205999552,13cyclictest323977-21modprobe23:00:013
31205999551,12cyclictest363-21snmpd19:21:143
31205699554,15cyclictest318740-21modprobe21:19:592
312054995528,23cyclictest23-21ksoftirqd/122:20:021
312054995528,23cyclictest23-21ksoftirqd/122:20:021
31205499552,15cyclictest317384-21latency_hist20:50:031
312050995538,13cyclictest235-21systemd-journal21:15:000
31205999544,4cyclictest0-21swapper/319:10:343
31205999543,6cyclictest325538-21munin-run23:30:023
31205999542,6cyclictest317702-21munin-run21:00:003
312059995424,25cyclictest235-21systemd-journal23:58:343
31205999542,15cyclictest321609-21munin-run22:15:003
31205999542,14cyclictest1-21systemd00:30:013
31205999542,12cyclictest363-21snmpd19:17:403
31205999542,12cyclictest313796-21sshd19:43:343
31205999542,12cyclictest313260-21sshd19:32:473
31205999542,11cyclictest1-21systemd20:05:013
31205999541,9cyclictest328366-21sshd00:22:363
31205999541,9cyclictest328366-21sshd00:22:363
31205499543,15cyclictest412-21sshd21:45:341
31205499542,15cyclictest322167-21unin-run22:25:001
312054995420,31cyclictest319947-21latency_hist21:40:021
312054995420,31cyclictest319947-21latency_hist21:40:021
312054995420,30cyclictest316551-21latency_hist20:35:011
312054995415,33cyclictest0-21swapper/120:15:011
312050995430,11cyclictest306-21in:imuxsock21:00:000
31205999532,6cyclictest320070-21munin-run21:45:003
31205999532,14cyclictest322583-21latency_hist22:30:013
31205699533,12cyclictest327368-21cat00:05:012
312056995325,6cyclictest30-21ksoftirqd/200:35:012
31205499532,15cyclictest363-21snmpd23:47:181
31205499532,15cyclictest325778-21munin-run23:35:011
312054995315,7cyclictest323792-21latency_hist22:55:021
312054995315,6cyclictest0-21swapper/119:54:591
31205999522,13cyclictest315632-21munin-run20:20:013
31205999522,12cyclictest316389-21sshd20:34:343
31205999521,11cyclictest363-21snmpd23:13:223
312056995217,7cyclictest320868-21systemctl22:00:002
31205499522,15cyclictest329208-21munin-run00:40:021
31205499522,15cyclictest329208-21munin-run00:40:021
31205499522,15cyclictest315915-21munin-run20:25:001
31205499522,14cyclictest235-21systemd-journal22:30:011
31205499522,12cyclictest313528-21cron19:39:591
31205499522,11cyclictest324473-21modprobe23:10:001
31205499522,10cyclictest327093-40mandb00:00:011
312050995225,17cyclictest312507-21run-parts19:20:000
312059995122,25cyclictest326797-21cron23:55:003
31205999511,9cyclictest315888-21cron20:25:003
31205999511,16cyclictest1-21systemd23:05:013
312056995117,9cyclictest30-21ksoftirqd/220:35:012
31205499514,14cyclictest272-21dbus-daemon19:15:001
31205499512,15cyclictest328391-21munin-run00:25:011
31205499512,15cyclictest328391-21munin-run00:25:001
31205499512,13cyclictest322673-21unin-run22:35:001
31205499512,13cyclictest318241-21modprobe21:10:001
312054995121,25cyclictest321578-21kworker/u8:222:14:591
312054995116,30cyclictest0-21swapper/121:20:021
312054995114,7cyclictest23-21ksoftirqd/123:24:351
31205999503,5cyclictest314316-21modprobe19:54:593
31205999502,6cyclictest318738-21cron21:20:003
31205999502,15cyclictest316254-21latency_hist20:30:033
312056995019,8cyclictest30-21ksoftirqd/221:25:032
312056995019,8cyclictest30-21ksoftirqd/221:25:022
31205499503,13cyclictest321494-21latency_hist22:10:031
31205499502,15cyclictest324712-21sshd23:13:341
312054995018,10cyclictest23-21ksoftirqd/119:25:021
312054995016,4cyclictest320959-21latency_hist22:00:021
312050995039,6cyclictest316646-21cron20:40:000
312050995033,13cyclictest235-21systemd-journal20:00:000
312050995020,26cyclictest12-21ksoftirqd/022:00:020
31205999492,8cyclictest322948-21true22:40:003
31205999491,15cyclictest313173-21latency_hist19:30:023
31205999491,12cyclictest363-21snmpd00:00:233
312056994915,10cyclictest325828-21sed23:35:022
31205499493,10cyclictest312502-21latency_hist19:20:011
312054994924,21cyclictest23-21ksoftirqd/121:45:001
31205499492,14cyclictest313288-21munin-run19:35:011
31205999482,7cyclictest321846-21sshd22:17:353
31205999482,7cyclictest321846-21sshd22:17:343
31205999481,11cyclictest363-21snmpd21:03:033
312056994816,8cyclictest324645-21date23:10:012
31205699481,34cyclictest291rcuc/219:35:022
312054994831,13cyclictest1-21systemd22:05:001
312054994823,22cyclictest319152-21latency_hist21:25:031
312054994823,22cyclictest319152-21latency_hist21:25:031
31205499482,15cyclictest323955-21modprobe23:00:011
31205499482,15cyclictest316156-21/usr/sbin/munin20:30:011
31205499481,13cyclictest363-21snmpd19:10:011
312050994828,16cyclictest327314-21sshd00:00:350
312050994827,15cyclictest281-21systemd-logind21:30:010
31205499472,8cyclictest327370-21wc00:05:021
312054994724,20cyclictest23-21ksoftirqd/100:35:011
31205499472,17cyclictest326810-21munin-run23:54:591
31205499471,15cyclictest320559-21sshd21:54:351
312054994711,5cyclictest0-21swapper/123:45:011
31205099475,30cyclictest320090-21latency_hist21:45:000
312050994735,9cyclictest12-21ksoftirqd/000:40:010
312050994735,9cyclictest12-21ksoftirqd/000:40:000
312050994729,14cyclictest235-21systemd-journal20:25:000
31205999467,34cyclictest0-21swapper/319:10:023
312056994628,12cyclictest235-21systemd-journal22:05:012
31205499462,15cyclictest315650-21sendmail-msp20:19:591
312054994617,5cyclictest314105-21latency_hist19:50:011
312054994614,4cyclictest315075-21kworker/u8:120:45:001
312054994614,4cyclictest315075-21kworker/u8:120:45:001
312050994629,13cyclictest235-21systemd-journal22:15:030
312050994629,13cyclictest235-21systemd-journal22:15:020
312050994622,17cyclictest235-21systemd-journal19:40:000
31205099461,15cyclictest317310-21latency_hist20:50:010
31205099454,37cyclictest235-21systemd-journal21:40:010
31205099454,37cyclictest235-21systemd-journal21:40:000
312050994530,7cyclictest306-21in:imuxsock20:20:010
312056994432,10cyclictest30-21ksoftirqd/219:50:022
312050994437,4cyclictest16950irq/224-800000019:23:150
312050994431,9cyclictest1-21systemd23:50:020
31205099443,12cyclictest316225-21latency_hist20:30:020
312050994430,9cyclictest306-21in:imuxsock20:52:560
312050994430,8cyclictest323953-21munin-plugin-st23:00:010
312050994421,16cyclictest235-21systemd-journal19:45:000
31205699434,4cyclictest131rcu_preempt22:20:012
31205699434,4cyclictest131rcu_preempt22:20:002
31205699432,8cyclictest316668-21munin-run20:40:002
312054994312,9cyclictest23-21ksoftirqd/120:58:351
312050994332,8cyclictest12-21ksoftirqd/023:20:020
312050994331,8cyclictest306-21in:imuxsock00:33:360
312050994331,10cyclictest12-21ksoftirqd/021:25:010
312050994331,10cyclictest12-21ksoftirqd/021:25:000
312050994329,10cyclictest235-21systemd-journal22:28:340
31205699424,4cyclictest131rcu_preempt20:00:002
31205699423,3cyclictest131rcu_preempt23:15:012
31205699421,4cyclictest131rcu_preempt00:10:032
31205499422,12cyclictest281-21systemd-logind00:30:011
312054994210,7cyclictest323647-21cat22:50:021
312050994232,6cyclictest19050irq/34-mmc100:30:000
312050994228,9cyclictest306-21in:imuxsock22:10:010
312050994226,11cyclictest235-21systemd-journal19:54:590
312050994222,14cyclictest12-21ksoftirqd/020:34:590
312050994216,15cyclictest314858-21munin-plugin-st20:05:000
312050994214,20cyclictest322144-21modprobe22:25:000
312056994122,14cyclictest272-21dbus-daemon19:40:012
312054994125,12cyclictest272-21dbus-daemon21:15:021
312054994116,5cyclictest313926-21cut19:45:021
312050994128,9cyclictest1-21systemd19:15:010
31205099412,7cyclictest363-21snmpd20:13:330
312050994126,6cyclictest306-21in:imuxsock23:25:010
31205699405,7cyclictest30-21ksoftirqd/219:15:032
31205699403,6cyclictest0-21swapper/219:55:012
312056994028,9cyclictest235-21systemd-journal20:10:002
31205699401,3cyclictest131rcu_preempt20:20:002
31205499407,5cyclictest131rcu_preempt00:09:301
31205499407,5cyclictest131rcu_preempt00:09:291
31205099406,24cyclictest328414-21latency_hist00:25:020
31205099406,24cyclictest328414-21latency_hist00:25:020
31205099403,8cyclictest235-21systemd-journal22:45:010
312050994029,9cyclictest12-21ksoftirqd/022:54:590
312050994026,10cyclictest363-21snmpd22:01:570
312050994025,11cyclictest12-21ksoftirqd/021:50:010
312050994024,10cyclictest306-21in:imuxsock23:55:010
312050994023,13cyclictest363-21snmpd20:41:400
312050994023,13cyclictest363-21snmpd20:41:390
312050994020,14cyclictest1-21systemd21:55:010
31205699394,4cyclictest131rcu_preempt23:55:022
31205699393,31cyclictest328743-21cat00:30:012
312056993926,10cyclictest317979-21systemd-run21:05:002
31205699392,3cyclictest131rcu_preempt20:25:012
31205099392,7cyclictest326308-21grep23:45:000
312050993927,8cyclictest12-21ksoftirqd/023:35:010
31205099392,5cyclictest324745-21munin-run23:15:020
312050993918,14cyclictest1-21systemd19:50:020
312056993823,11cyclictest318533-21sed21:15:012
31205099385,24cyclictest321608-21modprobe22:15:000
31205099383,30cyclictest322866-21sed22:35:030
312050993831,4cyclictest16950irq/224-800000000:11:450
31205099383,12cyclictest235-21systemd-journal23:10:010
312050993828,7cyclictest12-21ksoftirqd/021:10:020
312050993825,2cyclictest0-21swapper/021:18:350
312050993822,11cyclictest0-21swapper/022:50:020
312056993727,7cyclictest317417-21sshd20:52:562
312056993726,8cyclictest306-21in:imuxsock19:15:022
312056993725,9cyclictest30-21ksoftirqd/220:15:002
312050993722,12cyclictest315072-21sshd20:05:340
312050993722,11cyclictest363-21snmpd19:26:230
312056993627,7cyclictest30-21ksoftirqd/221:30:022
312056993626,7cyclictest318427-21cat21:10:032
312056993624,9cyclictest30-21ksoftirqd/220:45:012
312056993624,9cyclictest30-21ksoftirqd/220:45:002
31205699361,12cyclictest312776-21cron19:24:592
312054993623,9cyclictest272-21dbus-daemon00:14:591
31205099369,18cyclictest326025-21modprobe23:40:000
31205099363,6cyclictest317943-21sshd21:01:360
312050993629,4cyclictest16950irq/224-800000021:32:550
312050993627,6cyclictest325544-21latency_hist23:30:020
312050993625,8cyclictest12-21ksoftirqd/023:05:020
312050993623,10cyclictest12-21ksoftirqd/019:10:020
312056993527,5cyclictest314853-21latency_hist20:04:592
312056993526,7cyclictest30-21ksoftirqd/220:30:022
312056993525,7cyclictest326595-21sed23:50:012
312056993524,9cyclictest30-21ksoftirqd/222:54:352
312056993523,8cyclictest30-21ksoftirqd/200:21:302
312056993523,8cyclictest30-21ksoftirqd/200:21:292
312056993520,11cyclictest30-21ksoftirqd/222:42:342
312056993520,11cyclictest30-21ksoftirqd/222:05:022
312056993519,12cyclictest325088-21cat23:20:022
31205099357,23cyclictest313280-21modprobe19:35:000
312056993427,4cyclictest314293-21kworker/u8:221:40:012
312056993427,4cyclictest314293-21kworker/u8:221:40:002
312056993427,4cyclictest313140-21latency_hist19:30:022
312056993425,6cyclictest30-21ksoftirqd/221:40:022
312056993423,9cyclictest30-21ksoftirqd/219:41:272
312056993423,7cyclictest30-21ksoftirqd/221:35:012
312056993422,9cyclictest30-21ksoftirqd/221:46:532
31205699332,8cyclictest0-21swapper/223:30:012
312056993325,5cyclictest306-21in:imuxsock22:45:032
312056993318,11cyclictest272-21dbus-daemon22:25:502
31205699331,6cyclictest326321-21munin-run23:45:012
31205699315,20cyclictest235-21systemd-journal21:00:012
31205699312,7cyclictest0-21swapper/200:10:022
31205699312,7cyclictest0-21swapper/200:10:012
312056993121,7cyclictest30-21ksoftirqd/223:05:032
312056993119,9cyclictest30-21ksoftirqd/223:23:092
312056993021,5cyclictest312121-21latency_hist19:10:012
312056992917,9cyclictest272-21dbus-daemon22:15:002
312054992819,7cyclictest23-21ksoftirqd/120:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional