You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-21 - 08:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sun Dec 21, 2025 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1028568996714,48cyclictest275-21dbus-daemon00:10:002
102856899632,18cyclictest1037834-21systemd-run22:10:002
102856899631,13cyclictest1044594-21munin-run00:20:022
102856899631,13cyclictest1044594-21munin-run00:20:022
102856899622,19cyclictest1045608-21munin-run00:40:012
102856899622,16cyclictest1031871-21munin-run20:15:012
102856899622,14cyclictest1029809-21munin-run19:35:012
102856899612,54cyclictest1032409-21munin-run20:25:022
102856899612,54cyclictest1032409-21munin-run20:25:022
102856899612,13cyclictest1045058-21sshd00:28:162
102856899603,47cyclictest1035746-21cron21:29:592
102856899603,15cyclictest1044291-21munin-run00:15:002
102856899602,6cyclictest1031356-21munin-run20:05:002
1028568996024,32cyclictest1035231-21latency_hist21:20:012
102856899602,16cyclictest1037313-21sendmail22:00:012
102856899602,13cyclictest1030324-21munin-run19:45:002
102856899601,5cyclictest1037569-21munin-run22:05:002
102856899601,5cyclictest1034948-21sshd21:13:162
102856899601,16cyclictest1039113-21munin-run22:35:012
102856899601,13cyclictest1038085-21munin-run22:15:012
102856899593,49cyclictest1043784-21sh00:05:012
102856899592,5cyclictest1030574-21munin-run19:50:012
102856899592,5cyclictest1030574-21munin-run19:50:002
102856899592,52cyclictest1034167-21cron21:00:002
102856899591,9cyclictest1033391-21sshd20:40:162
102856899591,16cyclictest1036269-21munin-run21:40:012
1028568995825,28cyclictest30-21ksoftirqd/220:40:022
102856899582,48cyclictest1038319-21cron22:20:012
102856899582,11cyclictest1035487-21modprobe21:23:162
102856899582,11cyclictest1035487-21modprobe21:23:162
102856899581,5cyclictest1042955-21sshd23:45:162
102856899581,5cyclictest1042955-21sshd23:45:162
102856899581,5cyclictest1036772-21sshd21:49:162
102856899581,5cyclictest1031620-21munin-run20:10:022
102856899581,18cyclictest1039655-21munin-run22:45:012
102856899572,49cyclictest238-21systemd-journal21:55:012
102856899572,46cyclictest1040423-21modprobe23:00:002
102856899571,13cyclictest364-21snmpd23:32:522
102856199575,8cyclictest23-21ksoftirqd/120:59:591
102856899563,46cyclictest1032245-21grep20:20:012
102856899562,4cyclictest364-21snmpd21:41:302
102856899562,16cyclictest1041061-21grep23:10:022
102856899562,15cyclictest1034717-21munin-run21:10:012
102856899561,5cyclictest364-21snmpd19:37:272
102856899561,5cyclictest364-21snmpd19:21:022
102856899561,5cyclictest1041141-21sshd23:11:172
102856899561,5cyclictest1041141-21sshd23:11:162
1028568995615,34cyclictest1041953-21modprobe23:29:592
102856899561,17cyclictest1029614-21cat19:30:022
102856899561,17cyclictest1029614-21cat19:30:012
102856899561,12cyclictest364-21snmpd19:11:212
102856899561,10cyclictest1042470-21/usr/sbin/munin23:40:002
102856199563,8cyclictest23-21ksoftirqd/121:10:001
1028561995637,14cyclictest238-21systemd-journal00:00:001
102856199562,6cyclictest23-21ksoftirqd/120:25:011
102856199562,6cyclictest23-21ksoftirqd/120:25:001
102856899553,44cyclictest1029000-21cron19:19:592
102856899552,4cyclictest364-21snmpd22:46:432
102856899552,17cyclictest1030838-21systemd-run19:55:012
102856899551,5cyclictest364-21snmpd22:51:442
102856899551,4cyclictest1032660-21munin-run20:30:002
102856899551,11cyclictest364-21snmpd23:53:202
102856899551,10cyclictest364-21snmpd23:00:152
102856899544,44cyclictest0-21swapper/223:20:022
102856899542,5cyclictest364-21snmpd23:56:572
102856899542,11cyclictest364-21snmpd22:39:382
102856899541,6cyclictest364-21snmpd00:34:242
102856899533,45cyclictest1031065-21cron20:00:002
102856899531,3cyclictest364-21snmpd00:22:022
1028568995312,33cyclictest0-21swapper/223:44:592
102856199534,10cyclictest23-21ksoftirqd/123:15:011
102856199534,10cyclictest23-21ksoftirqd/123:15:011
102856899525,41cyclictest0-21swapper/222:30:022
102856899524,41cyclictest0-21swapper/220:35:022
102856899523,41cyclictest1038608-21modprobe22:24:592
102856899522,5cyclictest364-21snmpd20:51:192
1028561995214,9cyclictest23-21ksoftirqd/122:50:021
102856899512,44cyclictest1034597-21latency_hist21:05:022
102856899511,7cyclictest364-21snmpd23:24:252
102856899511,6cyclictest364-21snmpd20:47:482
102856899511,6cyclictest364-21snmpd20:47:482
102856199517,7cyclictest23-21ksoftirqd/119:35:011
102856199515,9cyclictest23-21ksoftirqd/122:25:011
102856199515,9cyclictest23-21ksoftirqd/119:25:021
102856199512,12cyclictest23-21ksoftirqd/119:30:011
102856199512,12cyclictest23-21ksoftirqd/119:30:001
1028555995142,6cyclictest12-21ksoftirqd/023:20:020
102856899503,40cyclictest1-21systemd21:35:012
102856199507,8cyclictest23-21ksoftirqd/121:30:001
102856199507,6cyclictest23-21ksoftirqd/120:00:021
102856199506,13cyclictest23-21ksoftirqd/121:20:021
1028561995028,16cyclictest275-21dbus-daemon20:05:011
102856199498,10cyclictest23-21ksoftirqd/120:34:591
1028573994831,12cyclictest275-21dbus-daemon00:00:013
102856199487,8cyclictest23-21ksoftirqd/121:04:171
102856199482,11cyclictest23-21ksoftirqd/100:10:001
102856199481,15cyclictest0-21swapper/100:34:161
102855599486,8cyclictest12-21ksoftirqd/019:40:010
102855599482,15cyclictest1038338-21munin-plugin-st22:20:000
1028561994734,10cyclictest23-21ksoftirqd/119:15:011
1028561994728,15cyclictest1-21systemd22:20:011
102856199471,13cyclictest23-21ksoftirqd/122:00:011
1028555994732,11cyclictest1-21systemd22:00:010
1028573994635,8cyclictest1044525-21sshd00:17:163
1028573994635,8cyclictest1044525-21sshd00:17:163
102856199468,13cyclictest23-21ksoftirqd/120:55:011
102856199464,8cyclictest23-21ksoftirqd/122:55:011
102856199463,7cyclictest23-21ksoftirqd/123:50:021
102856199463,7cyclictest23-21ksoftirqd/123:50:021
102856199463,12cyclictest23-21ksoftirqd/120:15:001
1028561994630,12cyclictest1-21systemd21:45:011
1028561994627,14cyclictest275-21dbus-daemon23:05:011
1028555994639,5cyclictest12-21ksoftirqd/022:40:020
1028555994627,13cyclictest238-21systemd-journal20:10:010
102856199457,11cyclictest23-21ksoftirqd/119:45:001
102856199455,10cyclictest23-21ksoftirqd/121:55:011
102856199453,11cyclictest23-21ksoftirqd/121:15:011
1028561994530,11cyclictest238-21systemd-journal23:22:001
1028561994526,13cyclictest238-21systemd-journal23:30:011
102856199452,15cyclictest23-21ksoftirqd/119:54:591
1028555994527,13cyclictest1-21systemd20:55:010
102855599452,39cyclictest0-21swapper/000:01:590
102856199444,13cyclictest23-21ksoftirqd/123:40:001
102856199442,8cyclictest1042741-21munin-run23:45:001
102856199441,9cyclictest1041392-21kworker/u8:300:16:161
102856199441,9cyclictest1041392-21kworker/u8:300:16:151
102856199441,12cyclictest1044907-21cut00:25:011
102856199441,12cyclictest0-21swapper/123:34:591
102855599449,31cyclictest12-21ksoftirqd/000:30:150
1028555994435,6cyclictest12-21ksoftirqd/022:59:170
102855599442,8cyclictest1-21systemd21:00:010
1028555994417,23cyclictest12-21ksoftirqd/022:50:020
102855599441,38cyclictest1030796-21sshd19:52:170
1028573994334,5cyclictest1035244-21sendmail21:20:013
1028573994331,9cyclictest1-21systemd21:05:013
1028573994327,12cyclictest238-21systemd-journal19:34:593
1028573994327,11cyclictest1030033-21cron19:39:593
102856899433,36cyclictest0-21swapper/219:10:022
102856199436,7cyclictest23-21ksoftirqd/120:45:011
102856199435,9cyclictest23-21ksoftirqd/123:10:011
1028561994335,6cyclictest23-21ksoftirqd/100:15:011
1028561994334,6cyclictest23-21ksoftirqd/120:50:011
1028561994334,6cyclictest23-21ksoftirqd/120:50:001
1028561994332,8cyclictest275-21dbus-daemon22:30:001
102856199433,11cyclictest23-21ksoftirqd/121:25:011
102856199433,11cyclictest23-21ksoftirqd/121:25:001
1028561994324,13cyclictest1-21systemd19:20:011
102855599436,9cyclictest12-21ksoftirqd/021:20:010
102855599436,34cyclictest1028551-21cyclictest21:40:030
102855599434,35cyclictest12-21ksoftirqd/020:20:010
1028555994331,9cyclictest12-21ksoftirqd/020:25:170
1028573994226,13cyclictest37-21ksoftirqd/322:20:003
1028573994225,12cyclictest275-21dbus-daemon22:10:013
1028573994225,12cyclictest238-21systemd-journal22:40:013
1028573994224,11cyclictest1-21systemd23:50:023
1028573994224,11cyclictest1-21systemd23:50:023
1028573994220,16cyclictest275-21dbus-daemon23:40:003
102856199424,9cyclictest23-21ksoftirqd/122:43:251
102856199423,9cyclictest23-21ksoftirqd/120:10:011
1028561994234,5cyclictest23-21ksoftirqd/122:14:591
1028561994221,15cyclictest275-21dbus-daemon00:30:011
102856199421,9cyclictest23-21ksoftirqd/122:35:021
1028555994233,6cyclictest19350irq/34-mmc100:00:020
1028555994232,6cyclictest1034714-21munin-plugin-st21:10:010
102855599422,7cyclictest12-21ksoftirqd/023:25:000
1028555994225,13cyclictest1044804-21sshd00:23:150
1028555994217,13cyclictest12-21ksoftirqd/021:53:150
1028573994127,9cyclictest275-21dbus-daemon00:35:013
1028573994127,10cyclictest1-21systemd00:30:013
1028573994123,13cyclictest37-21ksoftirqd/321:30:003
102856199414,9cyclictest23-21ksoftirqd/121:38:171
1028561994132,5cyclictest1043783-21sh00:05:011
1028561994126,11cyclictest23-21ksoftirqd/100:38:161
1028561994120,5cyclictest221rcuc/120:16:141
1028561994111,4cyclictest131rcu_preempt19:40:011
1028555994134,4cyclictest16950irq/224-800000023:10:000
1028555994134,4cyclictest16950irq/224-800000000:40:010
1028555994128,11cyclictest12-21ksoftirqd/023:40:000
1028555994127,8cyclictest1037556-21cron22:04:590
1028555994124,13cyclictest1044270-21sshd00:13:420
102855599411,5cyclictest1029008-21modprobe19:20:000
1028555994113,23cyclictest151rcuc/021:25:030
1028555994113,23cyclictest151rcuc/021:25:020
1028573994027,9cyclictest37-21ksoftirqd/319:50:023
1028573994027,9cyclictest37-21ksoftirqd/319:50:013
1028573994026,11cyclictest37-21ksoftirqd/322:55:013
1028573994023,13cyclictest275-21dbus-daemon19:55:013
102856199405,8cyclictest23-21ksoftirqd/120:30:001
1028561994031,7cyclictest23-21ksoftirqd/122:00:171
102855599408,5cyclictest131rcu_preempt21:36:150
1028555994028,9cyclictest12-21ksoftirqd/021:14:590
1028555994026,11cyclictest1038081-21systemd-run22:15:010
1028573993928,7cyclictest309-21in:imuxsock22:35:003
1028573993921,12cyclictest238-21systemd-journal20:30:003
1028573993919,15cyclictest275-21dbus-daemon20:50:013
1028573993919,15cyclictest275-21dbus-daemon20:50:003
1028573993916,19cyclictest275-21dbus-daemon22:00:013
102857399391,3cyclictest131rcu_preempt23:45:003
102856199395,6cyclictest23-21ksoftirqd/123:18:151
102856199394,10cyclictest23-21ksoftirqd/123:50:161
1028561993929,7cyclictest23-21ksoftirqd/121:50:011
102855599392,7cyclictest1033427-21munin-run20:45:010
1028555993926,9cyclictest12-21ksoftirqd/000:28:450
1028555993926,10cyclictest12-21ksoftirqd/019:29:170
1028555993926,10cyclictest12-21ksoftirqd/019:29:160
1028555993925,11cyclictest12-21ksoftirqd/022:10:010
1028555993925,11cyclictest12-21ksoftirqd/000:10:000
1028555993923,12cyclictest1029268-21sshd19:22:180
1028555993916,20cyclictest12-21ksoftirqd/023:01:460
102855599391,34cyclictest0-21swapper/021:05:010
1028573993829,6cyclictest1-21systemd23:10:013
1028573993821,12cyclictest275-21dbus-daemon22:45:013
1028573993819,13cyclictest275-21dbus-daemon23:30:013
1028573993818,15cyclictest1-21systemd23:55:003
1028573993818,14cyclictest275-21dbus-daemon22:05:003
1028573993817,15cyclictest275-21dbus-daemon21:35:013
102856199385,5cyclictest131rcu_preempt20:40:011
1028561993822,11cyclictest1028562-21latency_hist19:10:011
1028555993830,4cyclictest16950irq/224-800000019:59:590
1028555993828,7cyclictest12-21ksoftirqd/023:29:160
1028555993828,7cyclictest12-21ksoftirqd/020:50:010
1028555993828,7cyclictest12-21ksoftirqd/020:50:000
1028555993827,8cyclictest12-21ksoftirqd/019:45:020
1028555993825,9cyclictest309-21in:imuxsock21:34:160
1028555993824,11cyclictest12-21ksoftirqd/020:35:010
1028555993823,12cyclictest1031331-21sshd20:02:470
1028573993729,3cyclictest37-21ksoftirqd/320:55:003
1028573993726,9cyclictest37-21ksoftirqd/319:45:013
1028573993725,9cyclictest1044342-21sed00:15:023
1028573993724,10cyclictest37-21ksoftirqd/323:15:013
1028573993724,10cyclictest37-21ksoftirqd/323:15:003
102856199376,4cyclictest131rcu_preempt19:47:181
102856199376,4cyclictest131rcu_preempt19:47:181
1028561993724,10cyclictest238-21systemd-journal21:31:591
1028555993730,4cyclictest16950irq/224-800000020:40:000
1028555993730,4cyclictest12-21ksoftirqd/000:20:030
1028555993730,4cyclictest12-21ksoftirqd/000:20:030
102855599372,5cyclictest131rcu_preempt23:50:020
102855599372,5cyclictest131rcu_preempt23:50:020
102855599371,5cyclictest0-21swapper/022:35:010
102855599371,32cyclictest1028779-21ls19:15:010
102855599371,32cyclictest0-21swapper/021:50:020
1028573993629,5cyclictest37-21ksoftirqd/320:20:023
102857399362,6cyclictest131rcu_preempt21:55:013
1028573993623,10cyclictest37-21ksoftirqd/319:20:003
1028573993622,8cyclictest37-21ksoftirqd/319:15:003
102856199368,5cyclictest131rcu_preempt22:05:161
102855599368,5cyclictest131rcu_preempt22:54:160
102855599362,6cyclictest1042213-21munin-run23:35:000
1028555993624,9cyclictest12-21ksoftirqd/023:40:020
1028573993526,7cyclictest37-21ksoftirqd/321:40:003
102857399351,3cyclictest131rcu_preempt20:25:013
102857399351,3cyclictest131rcu_preempt20:25:003
102856199351,5cyclictest0-21swapper/122:39:541
1028555993528,5cyclictest12-21ksoftirqd/019:35:000
1028555993526,5cyclictest12-21ksoftirqd/019:49:170
1028555993526,5cyclictest12-21ksoftirqd/019:49:160
1028555993520,11cyclictest238-21systemd-journal22:45:000
1028555993519,13cyclictest12-21ksoftirqd/021:30:020
1028555993518,12cyclictest12-21ksoftirqd/022:25:010
102857399346,9cyclictest37-21ksoftirqd/323:34:313
1028573993426,5cyclictest309-21in:imuxsock22:48:173
1028573993423,8cyclictest37-21ksoftirqd/323:05:023
1028573993423,8cyclictest1031143-21sed20:00:013
1028573993420,9cyclictest0-21swapper/321:14:593
1028573993420,10cyclictest275-21dbus-daemon21:50:013
1028561993424,7cyclictest221rcuc/122:57:161
102857399334,3cyclictest131rcu_preempt00:22:013
102857399332,3cyclictest131rcu_preempt20:45:013
1028573993321,9cyclictest37-21ksoftirqd/319:30:033
1028573993321,9cyclictest37-21ksoftirqd/319:30:023
102855599334,4cyclictest131rcu_preempt20:15:010
1028555993323,7cyclictest12-21ksoftirqd/019:10:020
1028555993319,11cyclictest1043222-21sshd23:53:370
102855599331,28cyclictest0-21swapper/023:11:170
102855599331,28cyclictest0-21swapper/023:11:160
102857399323,24cyclictest1034182-21sendmail21:00:003
1028573993225,5cyclictest37-21ksoftirqd/319:23:183
1028573993224,6cyclictest37-21ksoftirqd/319:10:023
1028573993223,7cyclictest37-21ksoftirqd/323:20:023
1028573993219,10cyclictest1033085-21date20:35:023
1028573993214,14cyclictest275-21dbus-daemon22:30:013
1028555993221,9cyclictest12-21ksoftirqd/020:25:010
1028555993221,9cyclictest12-21ksoftirqd/020:25:000
102857399315,3cyclictest131rcu_preempt20:37:163
1028573993123,5cyclictest1043785-21latency_hist00:05:023
1028573993121,7cyclictest37-21ksoftirqd/320:10:033
1028573993120,5cyclictest1031854-21sh20:14:593
1028573993116,12cyclictest1-21systemd22:25:013
1028573993116,10cyclictest238-21systemd-journal21:10:013
1028573993115,11cyclictest1035502-21modprobe21:25:013
1028573993115,11cyclictest1035502-21modprobe21:25:003
1028555993117,10cyclictest309-21in:imuxsock22:30:010
1028573993018,9cyclictest37-21ksoftirqd/322:15:023
1028573993012,14cyclictest275-21dbus-daemon00:40:013
1028573992922,4cyclictest1041712-21munin-plugin-st23:24:593
1028573992919,4cyclictest37-21ksoftirqd/320:00:173
1028573992918,6cyclictest1044215-21latency_hist00:10:023
1028573992817,8cyclictest1036497-21date21:40:033
1028573992718,5cyclictest37-21ksoftirqd/322:58:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional