You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-26 - 16:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Tue Aug 26, 2025 12:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13037689910496,6cyclictest12-21ksoftirqd/012:30:010
130376899926,6cyclictest131rcu_preempt07:55:020
130376899692,14cyclictest1316453-21sshd11:12:360
130378199683,53cyclictest235-21systemd-journal11:55:033
130376899681,13cyclictest1315673-21sshd10:57:370
1303775996617,11cyclictest23-21ksoftirqd/107:20:021
1303775996617,11cyclictest23-21ksoftirqd/107:20:021
1303775996513,39cyclictest0-21swapper/109:25:021
130376899652,16cyclictest1307091-21munin-run08:15:010
130376899651,12cyclictest1317282-21munin-run11:30:000
1303775996415,8cyclictest23-21ksoftirqd/112:35:031
1303775996414,12cyclictest23-21ksoftirqd/109:50:021
130376899642,15cyclictest1315989-21munin-run11:05:010
130376899642,14cyclictest1316782-21munin-run11:20:010
130376899642,13cyclictest1318309-21munin-run11:50:000
1303768996419,41cyclictest1319059-21sshd12:01:220
130376899632,14cyclictest1314644-21sshd10:39:360
130376899631,17cyclictest1319322-21sshd12:08:360
130376899631,15cyclictest1305803-21munin-run07:50:010
1303775996215,11cyclictest23-21ksoftirqd/110:45:011
130376899621,13cyclictest363-21snmpd10:18:210
130376899612,13cyclictest1308417-21/usr/sbin/munin08:40:020
130376899611,5cyclictest1314425-21munin-run10:35:010
130376899611,15cyclictest363-21snmpd11:52:520
130376899602,14cyclictest1309173-21munin-run08:55:010
130376899601,13cyclictest1312777-21sshd10:00:370
130376899601,13cyclictest1312777-21sshd10:00:370
130376899601,12cyclictest1305010-21munin-run07:35:000
1303775995915,9cyclictest23-21ksoftirqd/109:50:021
1303768995922,33cyclictest0-21swapper/010:55:000
130376899592,13cyclictest1303937-21sshd07:11:370
130376899592,11cyclictest1317524-21munin-run11:35:000
130376899591,14cyclictest363-21snmpd09:01:400
130376899591,13cyclictest363-21snmpd10:45:040
1303775995817,9cyclictest23-21ksoftirqd/108:00:031
1303775995817,8cyclictest23-21ksoftirqd/111:40:021
1303775995817,8cyclictest23-21ksoftirqd/111:40:011
1303775995816,9cyclictest23-21ksoftirqd/108:35:021
1303775995812,11cyclictest23-21ksoftirqd/109:05:021
1303768995826,28cyclictest1311157-21sed09:30:020
1303768995822,32cyclictest1320894-21latency_hist12:40:010
1303768995822,32cyclictest1320894-21latency_hist12:40:010
130376899582,15cyclictest1304771-21munin-run07:30:010
1303768995726,26cyclictest1310883-21grep09:25:020
130376899572,15cyclictest1319737-21ls12:15:020
130376899571,13cyclictest1311215-21sshd09:30:510
130376899571,12cyclictest363-21snmpd08:34:360
1303775995615,8cyclictest23-21ksoftirqd/108:20:001
130376899562,16cyclictest1304213-21cron07:20:000
130376899562,16cyclictest1304213-21cron07:20:000
130376899562,15cyclictest1308878-21sshd08:46:360
1303768995615,36cyclictest272-21dbus-daemon11:25:020
1303768995615,36cyclictest272-21dbus-daemon11:25:010
130376899561,14cyclictest1306799-21sshd08:06:380
130376899561,13cyclictest1320148-21munin-run12:25:010
130376899561,13cyclictest1320148-21munin-run12:25:010
130376899561,12cyclictest363-21snmpd09:39:440
130377599558,14cyclictest1316491-21systemd-run11:15:001
1303775995515,8cyclictest23-21ksoftirqd/111:30:011
1303775995511,13cyclictest1-21systemd07:45:001
130376899554,17cyclictest1310554-21wc09:20:010
130376899552,15cyclictest1318778-21sshd11:56:200
130376899551,18cyclictest1306303-21munin-run08:00:000
130376899547,42cyclictest1-21systemd12:20:010
1303768995416,34cyclictest12-21ksoftirqd/010:21:400
130376899541,49cyclictest0-21swapper/007:45:010
130376899541,14cyclictest1308629-21sshd08:42:370
130376899541,13cyclictest363-21snmpd09:48:560
130376899541,12cyclictest1318030-21sshd11:42:370
130377599535,13cyclictest1-21systemd11:55:011
130376899533,5cyclictest1316325-21latency_hist11:10:020
1303768995332,18cyclictest1317810-21sh11:40:010
1303768995332,18cyclictest1317810-21sh11:40:000
130376899532,16cyclictest1320639-21munin-run12:35:000
130376899532,15cyclictest1309450-21systemctl09:00:000
130376899532,13cyclictest1309965-21munin-run09:10:020
130376899532,11cyclictest363-21snmpd09:59:430
1303768995318,32cyclictest12-21ksoftirqd/007:25:000
1303781995233,14cyclictest1-21systemd08:30:003
1303781995232,14cyclictest272-21dbus-daemon12:05:013
1303779995235,13cyclictest235-21systemd-journal09:30:022
130377599525,19cyclictest1304704-21cat07:25:021
1303775995238,10cyclictest1313848-21apt-get10:21:381
1303775995211,13cyclictest1307200-21sed08:15:011
130376899522,12cyclictest1314124-21sshd10:26:020
130376899521,13cyclictest1310184-21sshd09:14:370
1303775995133,13cyclictest235-21systemd-journal12:05:001
1303775995116,9cyclictest23-21ksoftirqd/109:35:011
130376899512,9cyclictest1312284-21cut09:54:590
130376899511,14cyclictest1303772-21cat07:10:010
130378199503,42cyclictest272-21dbus-daemon08:45:003
1303781995032,13cyclictest1313563-21cron10:20:013
130378199501,32cyclictest1308918-21kworker/3:209:35:033
1303779995028,15cyclictest1-21systemd09:40:002
1303768995013,33cyclictest1311950-21latency_hist09:45:020
1303768995011,9cyclictest235-21systemd-journal10:15:010
130376899501,13cyclictest1-21systemd08:25:000
130376899501,12cyclictest1306541-21sshd08:00:370
130376899501,11cyclictest363-21snmpd08:17:360
1303779994930,14cyclictest1-21systemd11:05:012
1303775994912,7cyclictest23-21ksoftirqd/111:50:011
1303775994912,6cyclictest23-21ksoftirqd/108:30:001
130376899492,11cyclictest1307868-21modprobe08:30:000
1303768994912,3cyclictest0-21swapper/007:40:010
1303781994829,15cyclictest235-21systemd-journal09:50:023
1303781994810,7cyclictest131rcu_preempt09:10:033
1303775994833,11cyclictest235-21systemd-journal12:20:001
130376899483,13cyclictest1315079-21latency_hist10:45:030
1303781994734,10cyclictest235-21systemd-journal12:25:003
1303781994734,10cyclictest235-21systemd-journal12:25:003
1303781994733,11cyclictest235-21systemd-journal07:25:003
130377999473,4cyclictest131rcu_preempt12:30:002
130376899472,12cyclictest1313071-21systemd10:10:000
1303781994636,8cyclictest37-21ksoftirqd/312:30:013
1303781994631,10cyclictest235-21systemd-journal07:15:003
1303781994630,11cyclictest37-21ksoftirqd/311:05:003
1303781994622,8cyclictest341irq_work/308:00:013
130377999464,36cyclictest1306281-21cron08:00:012
1303779994631,11cyclictest1-21systemd09:10:012
1303779994629,12cyclictest235-21systemd-journal10:09:592
1303779994629,12cyclictest1314702-21systemd-run10:40:002
1303779994629,11cyclictest1-21systemd08:05:012
1303775994634,8cyclictest23-21ksoftirqd/109:39:591
1303775994631,11cyclictest235-21systemd-journal12:00:011
1303775994630,12cyclictest363-21snmpd09:29:481
1303775994628,14cyclictest1308873-21sshd08:45:401
1303775994627,14cyclictest1-21systemd07:40:021
130377599461,15cyclictest1316781-21systemd-run11:20:021
1303781994531,9cyclictest306-21in:imuxsock11:20:033
1303781994531,9cyclictest306-21in:imuxsock11:20:023
1303781994530,12cyclictest1-21systemd08:20:013
1303781994530,10cyclictest272-21dbus-daemon10:40:003
1303781994529,10cyclictest1319863-21run-parts12:20:013
1303781994525,16cyclictest272-21dbus-daemon07:55:013
1303779994531,10cyclictest235-21systemd-journal08:35:002
1303775994531,10cyclictest1306802-21sshd08:07:371
1303781994422,15cyclictest235-21systemd-journal07:20:003
1303781994422,15cyclictest235-21systemd-journal07:20:003
1303779994435,6cyclictest30-21ksoftirqd/211:10:012
1303779994434,8cyclictest30-21ksoftirqd/210:50:012
1303779994433,6cyclictest306-21in:imuxsock09:05:002
1303779994428,12cyclictest1-21systemd08:45:002
1303779994426,14cyclictest272-21dbus-daemon08:15:002
1303779994425,12cyclictest235-21systemd-journal10:00:002
1303775994434,6cyclictest1314649-21cron10:40:001
1303775994430,11cyclictest1309396-21cron09:00:001
1303775994429,11cyclictest1312784-21sshd10:01:381
1303775994429,11cyclictest1312784-21sshd10:01:371
1303775994411,5cyclictest0-21swapper/112:40:011
1303775994411,5cyclictest0-21swapper/112:40:001
1303781994327,12cyclictest235-21systemd-journal09:55:563
130377999434,4cyclictest131rcu_preempt12:40:012
130377999434,4cyclictest131rcu_preempt12:40:002
130377999434,3cyclictest131rcu_preempt10:50:022
1303779994335,5cyclictest1310201-21munin-plugin-st09:15:002
1303779994329,11cyclictest30-21ksoftirqd/208:52:372
1303779994326,12cyclictest272-21dbus-daemon10:25:012
1303781994232,7cyclictest1304760-21grep07:30:003
1303781994226,11cyclictest272-21dbus-daemon12:40:023
1303781994226,11cyclictest272-21dbus-daemon12:40:013
1303779994229,10cyclictest272-21dbus-daemon10:30:012
1303779994228,10cyclictest272-21dbus-daemon09:45:002
1303779994227,11cyclictest1-21systemd07:15:012
1303779994224,13cyclictest272-21dbus-daemon07:40:012
1303775994232,5cyclictest1314398-21cron10:35:001
1303775994229,10cyclictest1315946-21sshd11:01:371
1303775994229,10cyclictest1310505-21munin-run09:20:011
1303775994228,11cyclictest1307589-21sshd08:21:371
1303775994228,10cyclictest1315458-21munin-run10:55:021
1303775994226,13cyclictest1316191-21sshd11:05:371
1303775994226,11cyclictest1317503-21sshd11:33:371
1303775994224,13cyclictest363-21snmpd12:24:001
1303775994224,13cyclictest363-21snmpd12:23:591
130378199416,5cyclictest131rcu_preempt08:06:373
1303781994127,9cyclictest1319598-21cut12:15:003
1303781994127,10cyclictest235-21systemd-journal07:40:013
1303781994127,10cyclictest235-21systemd-journal07:32:373
1303781994118,6cyclictest361rcuc/308:05:013
1303781994110,5cyclictest131rcu_preempt08:37:483
130377999413,4cyclictest131rcu_preempt09:50:012
1303779994128,10cyclictest1307912-21cat08:30:012
1303779994127,11cyclictest30-21ksoftirqd/207:25:372
1303779994127,10cyclictest235-21systemd-journal08:24:592
1303779994125,12cyclictest272-21dbus-daemon10:45:012
1303779994125,12cyclictest1310481-21sendmail09:20:002
130377999412,4cyclictest131rcu_preempt11:45:022
1303779994123,13cyclictest272-21dbus-daemon11:40:002
1303779994123,13cyclictest272-21dbus-daemon11:40:002
1303779994120,15cyclictest272-21dbus-daemon08:20:012
130377999411,3cyclictest131rcu_preempt11:00:002
1303775994129,9cyclictest1309923-21sshd09:08:361
1303775994128,10cyclictest363-21snmpd11:20:101
1303775994128,10cyclictest363-21snmpd11:20:101
1303775994127,9cyclictest48-21kcompactd010:56:231
1303775994114,22cyclictest1312761-21sed10:00:021
1303775994112,23cyclictest1304935-21latency_hist07:30:021
130377599411,11cyclictest23-21ksoftirqd/112:05:191
1303781994030,8cyclictest37-21ksoftirqd/311:40:033
1303781994030,8cyclictest37-21ksoftirqd/311:40:023
1303781994026,10cyclictest1-21systemd12:00:013
1303781994025,9cyclictest306-21in:imuxsock11:20:003
1303781994025,8cyclictest1315682-21cron11:00:003
1303781994025,11cyclictest37-21ksoftirqd/310:15:013
1303781994022,13cyclictest272-21dbus-daemon08:25:003
130377999405,4cyclictest131rcu_preempt12:25:012
130377999405,4cyclictest131rcu_preempt12:25:002
130377999405,4cyclictest131rcu_preempt09:30:002
1303779994028,9cyclictest1308590-21cut08:40:042
1303779994028,8cyclictest30-21ksoftirqd/209:00:012
130377999402,4cyclictest131rcu_preempt08:09:592
1303779994024,12cyclictest1316858-21sed11:20:012
130377999402,3cyclictest131rcu_preempt12:35:002
130377599402,8cyclictest23-21ksoftirqd/110:10:371
1303775994027,9cyclictest1320388-21munin-run12:29:591
1303775994027,10cyclictest363-21snmpd10:45:021
1303775994024,12cyclictest23-21ksoftirqd/108:40:011
1303775994022,14cyclictest363-21snmpd10:15:301
1303781993926,9cyclictest37-21ksoftirqd/310:55:003
1303781993926,9cyclictest235-21systemd-journal09:00:003
130377999394,3cyclictest131rcu_preempt12:15:002
130377999394,3cyclictest131rcu_preempt12:10:012
1303779993930,5cyclictest1318305-21latency_hist11:50:002
1303779993928,6cyclictest306-21in:imuxsock11:25:022
1303779993928,6cyclictest306-21in:imuxsock11:25:012
1303779993927,9cyclictest30-21ksoftirqd/207:35:032
1303779993927,8cyclictest30-21ksoftirqd/208:50:012
130377999392,4cyclictest131rcu_preempt09:54:362
130377999391,4cyclictest131rcu_preempt07:25:022
1303775993927,9cyclictest363-21snmpd12:10:211
1303775993927,8cyclictest48-21kcompactd008:42:061
1303775993926,9cyclictest48-21kcompactd009:13:441
1303775993926,9cyclictest1306057-21munin-run07:55:011
1303775993926,10cyclictest363-21snmpd10:09:551
1303775993926,10cyclictest363-21snmpd07:49:261
1303775993925,10cyclictest363-21snmpd10:29:351
1303775993924,10cyclictest306-21in:imuxsock07:34:371
1303775993920,15cyclictest363-21snmpd07:12:221
1303781993826,9cyclictest37-21ksoftirqd/310:35:013
1303781993824,10cyclictest235-21systemd-journal09:35:023
1303781993822,12cyclictest235-21systemd-journal11:44:593
1303781993821,12cyclictest235-21systemd-journal09:30:003
130377999385,3cyclictest131rcu_preempt12:20:002
130377999383,3cyclictest131rcu_preempt12:01:372
1303779993827,6cyclictest1318809-21sh12:00:002
130377999381,5cyclictest131rcu_preempt11:35:012
130377999381,3cyclictest131rcu_preempt11:55:002
1303775993827,7cyclictest1306315-21sendmail08:00:001
1303775993825,10cyclictest1309159-21munin-run08:55:011
1303781993724,8cyclictest306-21in:imuxsock10:10:003
1303781993714,17cyclictest272-21dbus-daemon11:10:013
1303779993725,9cyclictest272-21dbus-daemon11:30:002
1303779993723,10cyclictest272-21dbus-daemon07:55:012
1303775993725,9cyclictest48-21kcompactd009:41:281
1303775993725,9cyclictest1318022-21sshd11:41:151
130378199366,5cyclictest131rcu_preempt09:20:483
130378199362,8cyclictest0-21swapper/311:14:423
130378199362,4cyclictest131rcu_preempt12:35:013
130378199362,29cyclictest306-21in:imuxsock10:27:363
1303781993611,6cyclictest131rcu_preempt09:10:453
130377999364,4cyclictest131rcu_preempt10:35:002
1303779993625,8cyclictest1312799-21cron10:05:012
1303779993625,8cyclictest1312799-21cron10:05:002
1303779993622,9cyclictest272-21dbus-daemon07:50:012
130377999361,5cyclictest131rcu_preempt09:20:022
130378199355,3cyclictest131rcu_preempt10:05:003
130378199355,3cyclictest131rcu_preempt10:05:003
1303781993522,9cyclictest37-21ksoftirqd/309:05:013
130377999353,4cyclictest131rcu_preempt07:43:372
130377999352,3cyclictest131rcu_preempt10:20:002
1303779993520,11cyclictest0-21swapper/207:20:002
1303779993520,11cyclictest0-21swapper/207:20:002
130377599352,28cyclictest1303859-21cat07:10:011
130378199343,17cyclictest1313894-21cron10:25:003
1303781993423,8cyclictest37-21ksoftirqd/307:50:013
1303781993423,7cyclictest1-21systemd09:50:013
1303781993415,15cyclictest272-21dbus-daemon07:45:003
1303779993422,9cyclictest272-21dbus-daemon11:15:012
130377999341,3cyclictest131rcu_preempt07:10:012
1303781993325,5cyclictest1314938-21latency_hist10:45:023
1303781993318,11cyclictest1-21systemd12:10:003
130377999332,3cyclictest131rcu_preempt10:11:372
130378199323,4cyclictest131rcu_preempt09:20:003
1303781993225,4cyclictest1307074-21latency_hist08:15:023
1303781993222,8cyclictest37-21ksoftirqd/311:50:023
1303781993222,7cyclictest37-21ksoftirqd/311:30:013
1303781993221,7cyclictest306-21in:imuxsock10:50:003
130378199314,4cyclictest131rcu_preempt09:42:363
1303781993124,4cyclictest1317637-21latency_hist11:35:013
1303781993123,5cyclictest1308903-21latency_hist08:50:023
1303781993123,5cyclictest1303709-21latency_hist07:10:023
130378199284,19cyclictest1309161-21cut08:55:003
130378199281,3cyclictest131rcu_preempt08:34:383
nosleep12:21:514
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional