You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 03:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sat Dec 06, 2025 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1299308991052,101cyclictest0-21swapper/023:30:020
12993089910293,7cyclictest12-21ksoftirqd/023:55:030
1299313997030,36cyclictest1303715-21latency_hist20:35:011
129930899701,64cyclictest1309786-21cat22:30:020
129930899701,64cyclictest1299507-21sshd19:11:390
1299313996820,44cyclictest1312142-21sed23:15:021
1299313996820,44cyclictest1312142-21sed23:15:021
1299308996716,47cyclictest1307832-21sshd21:52:400
1299313996627,35cyclictest1300927-21latency_hist19:40:011
1299308996623,39cyclictest1306082-21/usr/sbin/munin21:20:020
129930899661,59cyclictest1316265-21sed00:35:020
129930899652,28cyclictest1309278-21date22:20:020
129930899651,60cyclictest1314054-21sshd23:50:410
129930899651,60cyclictest1314054-21sshd23:50:410
129930899651,60cyclictest1306283-21sshd21:20:390
129930899642,56cyclictest1307058-21sshd21:39:410
1299308996421,39cyclictest1315384-21systemctl00:19:280
129930899641,59cyclictest1315697-21sshd00:22:420
129930899641,2cyclictest0-21swapper/023:03:410
129931399637,14cyclictest272-21dbus-daemon23:45:001
129930899632,55cyclictest1-21systemd20:00:380
1299308996319,40cyclictest1312234-21sshd23:16:410
129930899631,16cyclictest1-21systemd19:30:000
1299313996217,41cyclictest1300031-21cat19:20:031
129930899623,55cyclictest1311979-21sshd23:11:410
129930899623,55cyclictest1311979-21sshd23:11:400
129930899622,56cyclictest1303922-21sshd20:37:400
129930899622,55cyclictest1300545-21sshd19:30:390
1299308996221,36cyclictest1301808-21latency_hist19:55:030
1299308996221,36cyclictest1301808-21latency_hist19:55:020
129930899621,10cyclictest1314857-21sshd00:07:400
129930899612,54cyclictest1310440-21sshd22:42:020
129930899611,8cyclictest1304448-21sshd20:46:360
129930899611,8cyclictest1304448-21sshd20:46:350
129930899611,6cyclictest1303677-21sshd20:33:410
129930899611,56cyclictest1302402-21munin-run20:10:010
129930899611,56cyclictest1302402-21munin-run20:10:010
129930899602,53cyclictest1311748-21cron23:10:000
129930899601,54cyclictest363-21snmpd22:06:090
129930899601,54cyclictest1313287-21sshd23:38:410
129930899601,53cyclictest1315138-21munin-run00:15:000
129931399595,49cyclictest1301864-21modprobe19:59:591
1299313995934,22cyclictest1311740-21sshd23:08:411
129931399592,13cyclictest1315470-21sendmail00:20:011
129930899592,53cyclictest1303395-21cat20:25:030
129930899592,52cyclictest1300064-21cron19:25:010
129930899591,8cyclictest1310688-21sshd22:45:390
129930899591,54cyclictest363-21snmpd21:47:550
129930899591,54cyclictest1313566-21sshd23:43:420
129930899591,53cyclictest363-21snmpd22:53:490
129930899591,53cyclictest1309951-21munin-run22:35:000
129930899591,53cyclictest1304881-21latency_hist20:55:020
129931399584,13cyclictest1315275-21latency_hist00:15:021
129930899585,47cyclictest1310201-21munin-plugin-st22:40:000
1299308995829,25cyclictest1301303-21latency_hist19:45:010
129930899582,52cyclictest1305486-21sshd21:06:390
129930899581,5cyclictest1312520-21sshd23:21:420
129930899581,53cyclictest363-21snmpd00:28:370
129930899581,52cyclictest1301346-21sshd19:48:390
129930899581,2cyclictest0-21swapper/021:15:020
1299315995739,14cyclictest1-21systemd00:20:012
129931399574,47cyclictest1313057-21munin-plugin-st23:35:001
129931399573,7cyclictest272-21dbus-daemon00:25:491
129931399572,7cyclictest1307594-21cron21:50:001
129931399572,13cyclictest1314151-21wc23:55:021
129931399572,13cyclictest1314151-21wc23:55:021
1299308995719,34cyclictest12-21ksoftirqd/022:05:010
129930899571,6cyclictest1313023-21sshd23:31:410
129930899571,52cyclictest363-21snmpd21:44:050
129930899571,51cyclictest363-21snmpd20:43:000
129930899571,51cyclictest363-21snmpd19:15:420
129930899571,51cyclictest1306803-21sshd21:33:390
129930899571,4cyclictest1305243-21sshd21:04:410
129930899571,4cyclictest1305243-21sshd21:04:400
129930899571,12cyclictest1306573-21/usr/sbin/munin21:30:010
1299313995614,38cyclictest1315761-21latency_hist00:25:011
1299308995627,26cyclictest1304969-21cron21:00:000
129930899562,50cyclictest363-21snmpd20:29:410
129930899562,49cyclictest363-21snmpd22:57:420
1299308995619,33cyclictest1316517-21modprobe00:40:000
129930899561,51cyclictest363-21snmpd23:46:470
129930899561,50cyclictest363-21snmpd22:23:510
129931399553,47cyclictest1314331-21cron00:00:001
129931399553,17cyclictest1305954-21latency_hist21:15:021
1299313995527,25cyclictest1307953-21latency_hist21:55:021
129931399552,15cyclictest1312792-21cron23:30:001
1299313995518,33cyclictest1305470-21latency_hist21:05:021
129930899552,48cyclictest363-21snmpd20:17:110
129931399546,3cyclictest131rcu_preempt19:50:021
129931399543,9cyclictest1308754-21latency_hist22:10:011
129931399542,16cyclictest1300084-21true19:25:001
129931399542,13cyclictest1311357-21latency_hist23:00:021
129931399542,13cyclictest1305373-21latency_hist21:05:021
1299313995417,33cyclictest272-21dbus-daemon20:30:011
1299313995412,37cyclictest1307341-21cron21:45:001
129931399541,11cyclictest1309282-21latency_hist22:20:011
129930899543,48cyclictest363-21snmpd00:03:540
129930899541,48cyclictest363-21snmpd20:12:010
1299315995310,19cyclictest1311982-21kworker/u8:200:05:032
129931399532,16cyclictest1299542-21true19:15:001
129931399532,14cyclictest1-21systemd19:45:011
129931399532,11cyclictest1303228-21latency_hist20:25:011
129931399531,13cyclictest1304927-21latency_hist20:55:021
1299313995310,6cyclictest1309142-21cron22:20:021
1299308995322,26cyclictest1-21systemd22:15:010
1299308995322,26cyclictest1-21systemd22:15:000
129930899531,47cyclictest363-21snmpd19:37:460
129930899531,2cyclictest0-21swapper/019:10:020
129931399529,3cyclictest0-21swapper/100:36:401
129931399522,9cyclictest1310748-21wc22:50:011
1299313995225,23cyclictest1305519-21latency_hist21:10:021
1299313995223,25cyclictest1309957-21latency_hist22:35:011
129931399522,11cyclictest1312275-21sendmail23:19:591
129931399521,9cyclictest1314871-21cron00:10:001
129930899521,47cyclictest363-21snmpd21:55:280
129931399513,7cyclictest235-21systemd-journal19:35:011
129931399512,9cyclictest1310964-21cron22:55:001
129931399512,8cyclictest1306306-21cron21:25:001
129931399512,8cyclictest1304214-21cron20:45:001
129931399512,7cyclictest1308098-21cron22:00:011
1299313995125,22cyclictest1313975-21latency_hist23:50:031
129931399512,12cyclictest1304481-21/usr/sbin/munin20:50:011
129931399512,12cyclictest1304481-21/usr/sbin/munin20:50:001
1299313995115,32cyclictest1304130-21sed20:40:031
1299313995114,33cyclictest1310161-21sshd22:37:011
129930899513,44cyclictest363-21snmpd19:57:020
1299318995033,11cyclictest1-21systemd21:40:013
129931599501,33cyclictest291rcuc/200:30:022
129931399502,9cyclictest1310490-21latency_hist22:45:011
129931399502,7cyclictest235-21systemd-journal21:40:011
129931399502,14cyclictest1302659-21true20:15:011
129931399501,11cyclictest1312559-21true23:25:011
129931899498,31cyclictest361rcuc/320:35:033
1299315994934,10cyclictest235-21systemd-journal19:20:012
1299315994929,15cyclictest1-21systemd22:35:012
1299315994911,7cyclictest131rcu_preempt23:00:012
129931399496,5cyclictest0-21swapper/120:10:011
129931399496,5cyclictest0-21swapper/120:10:001
129931399493,7cyclictest272-21dbus-daemon20:05:001
1299313994934,12cyclictest235-21systemd-journal19:51:401
1299313994934,12cyclictest235-21systemd-journal19:51:391
129931399492,18cyclictest1304991-21munin-run21:00:001
1299313994915,31cyclictest23-21ksoftirqd/119:26:381
1299313994915,29cyclictest1314639-21sh00:05:001
1299313994914,31cyclictest1309828-21latency_hist22:30:021
1299313994912,32cyclictest0-21swapper/123:01:411
129931399491,11cyclictest1308900-21systemd-run22:15:011
129931399491,11cyclictest1308900-21systemd-run22:15:011
129931399491,10cyclictest363-21snmpd21:15:451
1299315994831,14cyclictest1316473-21sshd00:39:412
129931399489,4cyclictest0-21swapper/121:33:441
129931399481,13cyclictest1313294-21cron23:40:001
1299318994729,13cyclictest272-21dbus-daemon22:15:013
1299318994729,13cyclictest272-21dbus-daemon22:15:003
129931899471,3cyclictest131rcu_preempt20:10:013
129931899471,3cyclictest131rcu_preempt20:10:013
129931599479,25cyclictest1312269-21modprobe23:20:002
1299315994739,6cyclictest30-21ksoftirqd/220:30:012
1299315994730,14cyclictest272-21dbus-daemon20:35:002
1299315994729,11cyclictest30-21ksoftirqd/219:59:592
129931399472,8cyclictest1316336-21wc00:35:021
129931399472,16cyclictest1306542-21sshd21:26:441
129931399472,11cyclictest1308380-21modprobe22:05:001
1299313994712,30cyclictest23-21ksoftirqd/120:16:411
1299318994631,11cyclictest37-21ksoftirqd/321:00:003
1299315994633,9cyclictest1-21systemd21:55:002
1299315994628,13cyclictest1-21systemd20:50:012
1299315994628,13cyclictest1-21systemd20:50:012
129931599462,4cyclictest131rcu_preempt22:15:012
129931599462,4cyclictest131rcu_preempt22:15:002
129931399463,10cyclictest1299452-21latency_hist19:10:031
129931899454,4cyclictest131rcu_preempt00:00:013
1299318994532,10cyclictest1-21systemd19:15:023
1299318994531,10cyclictest235-21systemd-journal23:00:003
1299318994529,12cyclictest1-21systemd21:05:013
1299318994529,12cyclictest1-21systemd21:05:003
1299318994431,10cyclictest1302896-21sshd20:19:403
1299318994424,14cyclictest272-21dbus-daemon19:25:013
1299318994332,8cyclictest1315955-21sshd00:27:133
129931599438,6cyclictest131rcu_preempt21:30:002
1299315994329,9cyclictest306-21in:imuxsock22:40:002
1299318994229,7cyclictest306-21in:imuxsock20:55:013
1299318994225,12cyclictest272-21dbus-daemon00:05:013
1299315994232,7cyclictest1306042-21sendmail21:20:012
1299315994229,10cyclictest1-21systemd22:20:012
1299315994228,11cyclictest1312818-21systemd-run23:30:012
1299315994220,16cyclictest272-21dbus-daemon20:55:002
1299318994125,10cyclictest1-21systemd20:00:013
1299315994127,11cyclictest1300820-21cron19:40:002
1299315994127,10cyclictest306-21in:imuxsock21:05:032
1299315994126,8cyclictest1307087-21grep21:39:592
1299315994121,15cyclictest272-21dbus-daemon21:15:002
129931899408,9cyclictest37-21ksoftirqd/322:19:593
1299318994025,11cyclictest235-21systemd-journal21:14:413
1299318994024,12cyclictest272-21dbus-daemon21:20:023
1299315994028,8cyclictest235-21systemd-journal23:10:002
1299315994027,9cyclictest272-21dbus-daemon20:20:002
1299315994026,8cyclictest306-21in:imuxsock23:59:592
1299315994024,12cyclictest235-21systemd-journal19:45:012
1299315994020,13cyclictest1304989-21cat21:00:002
1299318993928,6cyclictest1310153-21kworker/u8:122:40:013
1299318993926,9cyclictest235-21systemd-journal23:20:023
1299318993926,9cyclictest235-21systemd-journal00:35:033
1299318993926,10cyclictest1316318-21sed00:35:023
1299318993921,13cyclictest1-21systemd22:30:013
1299318993919,14cyclictest1-21systemd00:25:013
129931899391,3cyclictest131rcu_preempt21:25:023
1299315993930,6cyclictest306-21in:imuxsock23:50:012
1299315993924,11cyclictest235-21systemd-journal21:24:412
1299315993920,14cyclictest272-21dbus-daemon21:05:012
1299315993920,14cyclictest272-21dbus-daemon21:05:002
1299315993917,18cyclictest272-21dbus-daemon22:25:002
1299318993828,7cyclictest1312258-21sshd23:19:423
1299318993827,8cyclictest37-21ksoftirqd/319:45:003
1299318993825,10cyclictest272-21dbus-daemon21:10:003
1299318993824,11cyclictest272-21dbus-daemon20:50:013
1299318993824,11cyclictest272-21dbus-daemon20:50:003
1299318993824,10cyclictest272-21dbus-daemon19:20:003
1299318993823,10cyclictest272-21dbus-daemon22:50:003
1299318993822,12cyclictest235-21systemd-journal22:10:013
129931899381,6cyclictest131rcu_preempt20:45:003
129931599384,7cyclictest235-21systemd-journal00:35:002
1299315993826,9cyclictest30-21ksoftirqd/223:55:012
1299315993826,9cyclictest30-21ksoftirqd/223:55:012
1299315993825,9cyclictest1-21systemd20:40:012
1299315993825,10cyclictest1308691-21sed22:10:022
1299315993824,10cyclictest235-21systemd-journal22:45:012
1299315993818,15cyclictest235-21systemd-journal23:15:012
1299315993818,15cyclictest235-21systemd-journal23:15:002
129931899375,3cyclictest131rcu_preempt19:35:003
1299318993725,9cyclictest1312788-21sshd23:29:423
1299318993723,10cyclictest37-21ksoftirqd/323:35:023
1299318993722,11cyclictest1-21systemd22:45:013
1299318993719,14cyclictest272-21dbus-daemon19:50:013
129931899371,3cyclictest131rcu_preempt21:55:023
129931899371,3cyclictest131rcu_preempt19:30:003
1299315993726,6cyclictest306-21in:imuxsock20:44:592
1299315993724,9cyclictest1301337-21sshd19:46:382
1299315993724,10cyclictest30-21ksoftirqd/221:35:012
1299315993722,10cyclictest235-21systemd-journal19:30:012
1299315993721,11cyclictest235-21systemd-journal23:00:022
1299315993720,13cyclictest235-21systemd-journal00:05:032
129931899364,4cyclictest131rcu_preempt21:35:003
1299318993628,5cyclictest1-21systemd23:35:003
1299318993625,7cyclictest1313849-21kworker/3:023:55:013
1299318993625,7cyclictest1313849-21kworker/3:023:55:003
1299318993624,8cyclictest37-21ksoftirqd/300:15:003
1299318993623,9cyclictest1307360-21systemd-run21:45:013
1299318993622,10cyclictest37-21ksoftirqd/320:15:003
1299315993628,5cyclictest412-21sshd21:44:402
1299315993627,6cyclictest1313308-21sh23:39:592
129931599362,3cyclictest131rcu_preempt19:35:012
1299315993620,11cyclictest272-21dbus-daemon20:05:012
129931899354,3cyclictest131rcu_preempt23:01:383
129931899354,3cyclictest131rcu_preempt22:21:573
129931899353,4cyclictest131rcu_preempt00:20:003
129931899353,23cyclictest1310970-21modprobe22:55:003
1299318993526,5cyclictest1308166-21kworker/3:222:05:013
1299318993525,8cyclictest37-21ksoftirqd/321:50:013
1299318993524,9cyclictest37-21ksoftirqd/320:25:003
1299318993524,8cyclictest37-21ksoftirqd/321:30:013
1299318993524,8cyclictest1309902-21sshd22:30:393
1299318993523,9cyclictest37-21ksoftirqd/320:05:003
1299318993519,11cyclictest272-21dbus-daemon00:10:003
1299318993518,13cyclictest1-21systemd19:55:013
1299318993518,13cyclictest1-21systemd19:55:003
129931899351,3cyclictest131rcu_preempt23:45:013
1299318993421,10cyclictest37-21ksoftirqd/320:30:003
1299315993425,5cyclictest306-21in:imuxsock22:04:592
1299315993424,7cyclictest0-21swapper/219:55:022
1299315993424,7cyclictest0-21swapper/219:55:022
1299315993423,8cyclictest30-21ksoftirqd/223:45:012
1299315993423,8cyclictest1308268-21sed22:00:022
1299315993423,6cyclictest306-21in:imuxsock20:15:002
1299315993417,14cyclictest272-21dbus-daemon00:15:002
1299315993416,13cyclictest272-21dbus-daemon19:15:012
1299315993415,14cyclictest272-21dbus-daemon20:25:022
129931899334,4cyclictest131rcu_preempt23:11:403
129931899334,4cyclictest131rcu_preempt23:11:393
1299318993320,10cyclictest306-21in:imuxsock21:55:023
129931899331,3cyclictest131rcu_preempt20:40:003
1299315993322,9cyclictest30-21ksoftirqd/222:49:412
1299315993322,6cyclictest306-21in:imuxsock19:25:002
1299315993321,9cyclictest30-21ksoftirqd/223:25:002
1299315993320,7cyclictest306-21in:imuxsock21:50:002
129931899323,23cyclictest1311761-21munin-run23:10:003
1299318993223,6cyclictest1313849-21kworker/3:023:50:023
1299315993226,4cyclictest1299398-21latency_hist19:10:012
1299315993223,5cyclictest2-21kthreadd22:55:012
1299315993222,6cyclictest1302413-21systemctl20:10:012
1299315993222,6cyclictest1302413-21systemctl20:10:002
1299315993215,12cyclictest281-21systemd-logind00:25:002
129931599321,3cyclictest131rcu_preempt22:26:582
129931899311,3cyclictest131rcu_preempt19:38:393
1299318993022,5cyclictest1289555-21kworker/3:219:10:033
129931599293,21cyclictest1313059-21cut23:35:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional