You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-25 - 23:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Tue Nov 25, 2025 12:45:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3302999692,2cyclictest0-21swapper/009:20:010
3314999064,22cyclictest37-21ksoftirqd/312:10:023
3312998471,9cyclictest30-21ksoftirqd/207:25:022
330299765,4cyclictest131rcu_preempt07:10:020
330299732,9cyclictest8036-21wc08:40:020
330299652,14cyclictest13430-21munin-run10:25:000
3302996521,40cyclictest12-21ksoftirqd/011:34:590
3314996316,12cyclictest37-21ksoftirqd/308:30:023
3314996316,12cyclictest37-21ksoftirqd/308:30:013
330299632,16cyclictest10302-21sshd09:25:000
330299632,16cyclictest10302-21sshd09:25:000
330299631,57cyclictest0-21swapper/012:17:400
330299631,14cyclictest14701-21sshd10:49:300
330799629,31cyclictest0-21swapper/107:50:021
3302996226,33cyclictest0-21swapper/012:06:300
330299622,15cyclictest8491-21munin-run08:50:010
330299622,12cyclictest4305-21sshd07:29:290
330299622,12cyclictest4305-21sshd07:29:280
330299618,4cyclictest5842-21sshd07:58:290
330299618,4cyclictest5842-21sshd07:58:280
3302996157,2cyclictest0-21swapper/008:05:020
3302996129,29cyclictest12-21ksoftirqd/008:15:010
330299612,7cyclictest7174-21cron08:25:000
330299612,13cyclictest11054-21sshd09:39:280
3312996041,15cyclictest1-21systemd09:40:012
3307996040,15cyclictest1-21systemd09:30:001
330299602,14cyclictest363-21snmpd12:27:190
330299602,13cyclictest16360-21munin-run11:20:010
330299602,12cyclictest3915-21latency_hist07:20:010
330299602,12cyclictest363-21snmpd12:35:130
330299601,8cyclictest13100-21latency_hist10:15:020
330299601,4cyclictest0-21swapper/011:20:240
330299592,13cyclictest4870-21munin-run07:40:010
330299592,13cyclictest18682-21munin-run12:05:010
3302995921,35cyclictest12-21ksoftirqd/008:55:000
330299592,12cyclictest363-21snmpd12:24:450
3302995913,38cyclictest0-21swapper/007:55:000
330299591,13cyclictest19181-21/usr/sbin/munin12:15:020
330299591,13cyclictest13131-21sshd10:18:290
330299591,13cyclictest13131-21sshd10:18:280
331499585,32cyclictest3746-21kworker/u8:007:25:023
3312995851,3cyclictest30-21ksoftirqd/208:05:022
330299582,15cyclictest3536-21systemd-run07:15:000
3302995821,31cyclictest1-21systemd09:00:020
330299582,10cyclictest13921-21cron10:35:010
330299581,52cyclictest9250-21sshd09:04:570
330299581,12cyclictest363-21snmpd11:05:300
331499572,8cyclictest0-21swapper/311:40:003
3302995731,23cyclictest1-21systemd10:30:000
330299572,7cyclictest16056-21munin-run11:15:010
330299572,6cyclictest15565-21munin-run11:05:020
330299572,6cyclictest14467-21munin-run10:45:010
330299571,14cyclictest5084-21sshd07:43:300
330299571,14cyclictest5084-21sshd07:43:290
330299571,13cyclictest11326-21sshd09:42:290
330299569,4cyclictest0-21swapper/010:00:470
330299562,8cyclictest10790-21sshd09:31:290
330299562,16cyclictest17343-21cron11:40:000
330299562,14cyclictest10569-21/usr/sbin/munin09:29:590
330299561,14cyclictest9751-21sshd09:12:290
330299561,12cyclictest48-21kcompactd011:50:040
330299561,12cyclictest4554-21sshd07:34:300
330299552,7cyclictest17850-21sshd11:45:300
3302995526,25cyclictest12-21ksoftirqd/008:30:010
3302995526,25cyclictest12-21ksoftirqd/008:30:000
330299552,17cyclictest6898-21cron08:19:590
330299552,15cyclictest363-21snmpd08:40:440
330299552,11cyclictest20215-21munin-run12:35:000
330299551,9cyclictest363-21snmpd11:29:590
3302995513,38cyclictest12148-21systemctl10:00:010
330299551,12cyclictest363-21snmpd10:37:000
330299551,10cyclictest11570-21sshd09:45:290
331499546,8cyclictest37-21ksoftirqd/310:10:013
330299541,49cyclictest0-21swapper/010:59:300
330299541,12cyclictest12635-21sh10:09:590
330299541,11cyclictest9493-21sshd09:08:280
330299532,14cyclictest272-21dbus-daemon10:55:000
330299522,10cyclictest18364-21sshd11:57:290
330299511,9cyclictest7657-21sshd08:30:290
330299511,8cyclictest17648-21munin-run11:45:020
330799503,29cyclictest4791-21kworker/u8:208:10:021
330299502,7cyclictest4059-21cron07:25:000
331499497,8cyclictest37-21ksoftirqd/310:39:593
331499494,15cyclictest37-21ksoftirqd/311:10:023
3307994935,10cyclictest23-21ksoftirqd/107:55:001
331499485,7cyclictest37-21ksoftirqd/310:00:023
331299489,24cyclictest3295-21cyclictest11:35:022
3302994814,30cyclictest5544-21latency_hist07:50:020
3314994717,21cyclictest361rcuc/309:38:283
3314994717,21cyclictest361rcuc/308:07:283
331499471,5cyclictest0-21swapper/311:30:033
3312994729,13cyclictest4326-21grep07:30:012
3312994729,13cyclictest4326-21grep07:30:002
330299471,10cyclictest6378-21sshd08:09:280
330299471,10cyclictest11831-21sshd09:52:270
331499465,9cyclictest37-21ksoftirqd/309:30:003
3314994625,15cyclictest235-21systemd-journal10:05:013
331499461,11cyclictest6035-21latency_hist08:00:033
331499461,11cyclictest6035-21latency_hist08:00:023
3312994631,11cyclictest272-21dbus-daemon08:15:002
3307994636,7cyclictest23-21ksoftirqd/110:20:011
3307994636,7cyclictest23-21ksoftirqd/110:20:011
3307994636,7cyclictest23-21ksoftirqd/107:20:011
3314994529,11cyclictest272-21dbus-daemon11:50:013
331499451,8cyclictest37-21ksoftirqd/309:20:003
3307994531,10cyclictest235-21systemd-journal10:20:281
3307994528,12cyclictest363-21snmpd09:45:021
330799451,7cyclictest1-21systemd10:45:011
331499447,12cyclictest37-21ksoftirqd/307:30:303
331499446,10cyclictest0-21swapper/312:20:003
331499445,7cyclictest37-21ksoftirqd/308:40:013
3314994433,7cyclictest361rcuc/311:19:593
3312994431,10cyclictest306-21in:imuxsock08:55:282
3312994429,11cyclictest235-21systemd-journal12:40:012
3307994429,11cyclictest235-21systemd-journal08:50:001
331499436,11cyclictest37-21ksoftirqd/310:59:263
3314994335,6cyclictest37-21ksoftirqd/309:20:033
3314994335,6cyclictest37-21ksoftirqd/309:20:033
3314994330,8cyclictest235-21systemd-journal11:25:013
3314994323,14cyclictest272-21dbus-daemon11:35:003
3314994311,6cyclictest131rcu_preempt12:40:013
3314994310,5cyclictest131rcu_preempt08:20:293
331299434,31cyclictest291rcuc/208:20:022
3312994331,9cyclictest9538-21pluginstate09:10:022
3312994323,15cyclictest272-21dbus-daemon09:50:012
3307994331,9cyclictest15200-21sshd10:57:301
3307994330,9cyclictest0-21swapper/108:39:591
3307994327,12cyclictest306-21in:imuxsock10:15:011
3307994327,11cyclictest272-21dbus-daemon09:10:011
3307994322,15cyclictest235-21systemd-journal07:30:011
3307994322,15cyclictest235-21systemd-journal07:30:011
3307994315,6cyclictest131rcu_preempt08:20:011
331499427,10cyclictest37-21ksoftirqd/307:19:363
331499422,9cyclictest37-21ksoftirqd/312:00:023
3314994227,12cyclictest13436-21systemd-run10:25:003
331499421,6cyclictest0-21swapper/308:15:013
331299423,5cyclictest131rcu_preempt08:45:012
3312994228,10cyclictest235-21systemd-journal12:10:012
330799426,8cyclictest131rcu_preempt07:10:021
330799426,6cyclictest131rcu_preempt11:40:001
3307994234,5cyclictest23-21ksoftirqd/111:20:021
3307994230,8cyclictest23-21ksoftirqd/107:25:011
3307994227,11cyclictest235-21systemd-journal11:59:301
3307994215,22cyclictest221rcuc/110:55:021
331499417,10cyclictest37-21ksoftirqd/309:54:283
331499415,9cyclictest37-21ksoftirqd/310:20:033
3314994133,6cyclictest37-21ksoftirqd/307:55:023
3314994130,7cyclictest9251-21cron09:05:003
3314994129,10cyclictest37-21ksoftirqd/312:25:013
3314994127,10cyclictest1-21systemd08:50:013
3314994120,6cyclictest361rcuc/310:41:303
3312994130,8cyclictest4580-21systemd-run07:35:012
3312994127,10cyclictest272-21dbus-daemon10:50:012
3312994125,12cyclictest30-21ksoftirqd/210:59:272
331299412,3cyclictest131rcu_preempt11:05:032
3312994121,15cyclictest272-21dbus-daemon09:05:012
3307994129,8cyclictest23-21ksoftirqd/108:15:001
3307994126,11cyclictest235-21systemd-journal07:56:301
3307994126,11cyclictest235-21systemd-journal07:56:301
3307994125,12cyclictest23-21ksoftirqd/109:40:001
3307994124,12cyclictest235-21systemd-journal11:55:001
330799411,7cyclictest0-21swapper/112:38:401
330799411,5cyclictest131rcu_preempt07:35:001
3307994111,5cyclictest131rcu_preempt08:25:301
3307994111,5cyclictest131rcu_preempt08:25:291
331499407,9cyclictest37-21ksoftirqd/311:05:033
331499407,8cyclictest37-21ksoftirqd/310:50:013
3314994030,7cyclictest37-21ksoftirqd/307:40:013
3314994028,9cyclictest6129-21sshd08:04:453
3314994025,11cyclictest37-21ksoftirqd/312:15:013
3314994023,12cyclictest1-21systemd10:55:003
331499401,13cyclictest0-21swapper/310:11:433
331499401,12cyclictest0-21swapper/308:43:183
3312994032,5cyclictest271-21cron10:20:012
3312994032,5cyclictest271-21cron10:20:002
3312994029,8cyclictest7666-21sshd08:32:282
3312994029,8cyclictest19950-21cron12:29:592
3312994029,8cyclictest13399-21sshd10:20:292
3312994024,12cyclictest8715-21sshd08:54:292
3312994024,12cyclictest235-21systemd-journal08:00:012
3312994024,12cyclictest235-21systemd-journal08:00:002
3312994023,12cyclictest272-21dbus-daemon11:50:002
3307994028,9cyclictest23-21ksoftirqd/109:55:021
3307994027,10cyclictest272-21dbus-daemon12:10:011
3307994026,11cyclictest23-21ksoftirqd/112:05:011
3307994023,13cyclictest272-21dbus-daemon12:25:501
330799401,6cyclictest0-21swapper/111:25:291
331499398,5cyclictest131rcu_preempt08:15:293
331499397,9cyclictest37-21ksoftirqd/312:30:013
3314993929,7cyclictest37-21ksoftirqd/309:50:023
3314993928,9cyclictest37-21ksoftirqd/311:10:223
3314993927,7cyclictest37-21ksoftirqd/307:15:023
3314993926,10cyclictest37-21ksoftirqd/308:53:283
3312993927,9cyclictest30-21ksoftirqd/209:45:012
3312993924,10cyclictest272-21dbus-daemon10:05:012
3312993921,13cyclictest235-21systemd-journal10:40:002
330799399,10cyclictest23-21ksoftirqd/110:35:011
3307993928,7cyclictest23-21ksoftirqd/111:50:011
3307993925,11cyclictest13690-21systemd-run10:30:011
3307993924,11cyclictest235-21systemd-journal11:35:001
330799391,8cyclictest0-21swapper/112:20:401
3307993917,19cyclictest23-21ksoftirqd/109:05:001
330799391,6cyclictest131rcu_preempt07:40:001
3307993910,5cyclictest131rcu_preempt10:00:021
3314993826,7cyclictest9504-21cron09:09:593
3314993821,12cyclictest37-21ksoftirqd/308:35:013
331499381,5cyclictest0-21swapper/309:33:463
331299388,7cyclictest30-21ksoftirqd/208:22:282
331299383,23cyclictest291rcuc/209:30:022
3312993825,9cyclictest3779-21cron07:19:592
3312993823,11cyclictest30-21ksoftirqd/210:15:012
3312993822,11cyclictest272-21dbus-daemon07:40:012
3312993821,12cyclictest1-21systemd11:40:002
3312993818,15cyclictest18397-21systemctl12:00:012
3312993818,14cyclictest1-21systemd09:35:002
331299381,3cyclictest131rcu_preempt12:05:002
3307993824,11cyclictest272-21dbus-daemon09:20:011
3307993824,10cyclictest306-21in:imuxsock12:16:591
3307993817,16cyclictest131rcu_preempt10:40:001
330799381,6cyclictest0-21swapper/111:04:301
330799381,5cyclictest0-21swapper/111:13:301
330799381,33cyclictest23-21ksoftirqd/108:45:021
331499375,5cyclictest131rcu_preempt09:15:013
3314993724,8cyclictest37-21ksoftirqd/311:55:023
331499371,8cyclictest0-21swapper/310:25:133
3314993710,4cyclictest131rcu_preempt08:57:303
3312993729,5cyclictest12112-21sendmail09:59:592
3312993728,6cyclictest19164-21sh12:15:002
3312993722,10cyclictest1-21systemd07:45:012
3312993722,10cyclictest1-21systemd07:45:002
3312993721,12cyclictest16038-21sshd11:13:302
3312993720,12cyclictest272-21dbus-daemon12:20:022
3307993728,6cyclictest23-21ksoftirqd/108:05:001
3307993726,7cyclictest8700-21sshd08:51:291
3307993724,9cyclictest23-21ksoftirqd/110:10:011
3307993723,10cyclictest23-21ksoftirqd/109:00:011
3307993712,6cyclictest131rcu_preempt10:05:011
331499365,4cyclictest131rcu_preempt07:30:013
331499365,4cyclictest131rcu_preempt07:30:003
3314993625,9cyclictest37-21ksoftirqd/311:45:033
3312993627,7cyclictest30-21ksoftirqd/212:35:022
3312993624,6cyclictest306-21in:imuxsock09:20:002
3312993624,10cyclictest30-21ksoftirqd/208:50:012
3312993623,8cyclictest306-21in:imuxsock11:29:312
3312993622,9cyclictest30-21ksoftirqd/211:55:012
331299361,3cyclictest131rcu_preempt07:50:002
3312993610,4cyclictest0-21swapper/209:15:012
3307993629,4cyclictest19165-21latency_hist12:15:011
3307993627,7cyclictest16569-21sshd11:21:301
3307993625,8cyclictest235-21systemd-journal09:23:261
3307993625,8cyclictest235-21systemd-journal09:23:251
330799362,28cyclictest23-21ksoftirqd/112:35:021
3307993621,10cyclictest272-21dbus-daemon08:25:011
331499354,7cyclictest37-21ksoftirqd/307:10:023
331499353,9cyclictest37-21ksoftirqd/310:30:163
3314993527,4cyclictest412-21sshd12:30:463
3314993522,8cyclictest306-21in:imuxsock07:45:013
3314993522,8cyclictest306-21in:imuxsock07:45:003
331499351,6cyclictest0-21swapper/310:15:443
331299353,3cyclictest131rcu_preempt09:55:012
3312993527,6cyclictest30-21ksoftirqd/208:40:012
3312993527,5cyclictest3249-21latency_hist07:10:012
331299352,3cyclictest131rcu_preempt09:20:032
331299352,3cyclictest131rcu_preempt09:20:022
3312993522,9cyclictest30-21ksoftirqd/210:45:012
3312993516,13cyclictest272-21dbus-daemon11:10:012
3307993527,6cyclictest23-21ksoftirqd/107:15:011
3307993525,8cyclictest23-21ksoftirqd/110:50:001
3307993521,10cyclictest235-21systemd-journal07:45:011
3307993521,10cyclictest235-21systemd-journal07:45:001
331499341,6cyclictest0-21swapper/307:47:303
3312993425,6cyclictest306-21in:imuxsock10:35:002
3312993423,9cyclictest30-21ksoftirqd/211:25:002
3312993422,9cyclictest272-21dbus-daemon08:30:012
3312993422,9cyclictest272-21dbus-daemon08:30:012
330799346,6cyclictest131rcu_preempt09:15:031
3307993425,6cyclictest23-21ksoftirqd/109:45:161
3307993417,13cyclictest272-21dbus-daemon08:35:011
330799341,4cyclictest131rcu_preempt11:10:021
3314993323,6cyclictest11332-21sshd09:44:293
331299337,3cyclictest131rcu_preempt08:10:002
331299333,23cyclictest17633-21munin-plugin-st11:45:012
3312993325,5cyclictest5634-21latency_hist07:55:022
3312993325,5cyclictest3538-21sh07:15:012
3312993322,8cyclictest30-21ksoftirqd/210:55:012
3312993314,15cyclictest272-21dbus-daemon10:10:002
3307993324,7cyclictest23-21ksoftirqd/111:45:011
331299322,4cyclictest131rcu_preempt11:20:002
3312993224,5cyclictest19852-21latency_hist12:25:032
3312993222,8cyclictest13659-21sshd10:27:282
3307993218,10cyclictest306-21in:imuxsock09:32:291
331499301,5cyclictest0-21swapper/312:02:083
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional