You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-10 - 03:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed Jul 09, 2025 12:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3723323999489,3cyclictest0-21swapper/008:45:010
3723333996718,11cyclictest30-21ksoftirqd/209:00:012
3723323996324,34cyclictest3738616-21munin-run12:05:020
3723323996224,4cyclictest3727171-21munin-run08:25:010
372333399617,14cyclictest3734264-21latency_hist10:40:012
3723323996050,7cyclictest3725598-21modprobe07:55:000
3723323996015,13cyclictest3727927-21latency_hist08:40:020
3723323995911,16cyclictest3727673-21latency_hist08:35:010
372333399588,15cyclictest3734025-21sed10:35:022
3723323995821,32cyclictest0-21swapper/007:45:010
3723323995820,4cyclictest3735485-21munin-run11:05:000
3723323995813,16cyclictest3731711-21cat09:50:010
3723329995738,14cyclictest1-21systemd11:25:011
3723333995616,11cyclictest30-21ksoftirqd/208:50:032
3723333995616,11cyclictest30-21ksoftirqd/208:50:022
3723337995529,20cyclictest272-21dbus-daemon09:50:003
372333399557,13cyclictest1-21systemd11:10:002
3723333995532,18cyclictest3733413-21munin-run10:25:012
3723333995532,18cyclictest3733413-21munin-run10:25:002
3723333995517,9cyclictest30-21ksoftirqd/210:15:012
3723323995536,15cyclictest3738038-21sshd11:51:230
372333399549,15cyclictest3725914-21latency_hist08:00:012
3723333995415,9cyclictest30-21ksoftirqd/207:15:012
3723333995332,16cyclictest1-21systemd07:50:012
3723323995343,7cyclictest3724031-21sshd07:22:250
372332399533,8cyclictest3740404-21latency_hist12:40:000
3723323995338,11cyclictest3724540-21sshd07:33:320
3723323995239,10cyclictest3727404-21sshd08:29:240
3723323995237,12cyclictest3734150-21sshd10:39:240
3723323995236,13cyclictest3729206-21sshd09:00:240
3723323995235,14cyclictest3731031-21sshd09:38:580
3723323995235,14cyclictest3731031-21sshd09:38:570
3723337995133,14cyclictest1-21systemd11:05:013
3723333995134,14cyclictest3729222-21sshd09:03:492
372332399518,39cyclictest0-21swapper/011:15:020
3723323995144,4cyclictest16950irq/224-800000010:05:000
3723323995135,13cyclictest3734915-21sshd10:50:230
3723323995135,13cyclictest3728675-21sshd08:50:240
3723323995135,13cyclictest3728675-21sshd08:50:240
3723323995134,13cyclictest3729725-21sshd09:12:240
3723333995031,15cyclictest3726855-21cron08:17:002
3723323995041,6cyclictest3737796-21sshd11:49:210
3723323995037,10cyclictest3739862-21sshd12:25:230
3723323995036,10cyclictest3730779-21sshd09:33:560
3723323995036,10cyclictest3726362-21sshd08:09:250
3723323995036,10cyclictest3725358-21munin-run07:50:010
3723323995035,12cyclictest3730261-21sshd09:20:230
3723323995034,13cyclictest3733880-21sshd10:30:230
3723337994930,14cyclictest235-21systemd-journal07:35:003
3723333994933,12cyclictest235-21systemd-journal08:35:002
3723333994929,16cyclictest3730000-21sshd09:19:232
3723323994937,9cyclictest3728944-21sshd08:59:240
3723323994936,10cyclictest3732594-21sshd10:08:240
3723323994935,11cyclictest3728435-21sshd08:48:480
3723323994935,10cyclictest3737008-21sshd11:33:240
3723323994935,10cyclictest3730562-21munin-run09:30:000
3723323994935,10cyclictest3726107-21sshd08:03:380
3723323994934,10cyclictest3738867-21munin-run12:10:010
3723333994833,11cyclictest1-21systemd11:20:022
3723333994831,13cyclictest3723754-21sshd07:18:242
3723333994830,13cyclictest272-21dbus-daemon12:15:002
372332999489,29cyclictest0-21swapper/110:05:011
3723329994828,14cyclictest272-21dbus-daemon09:00:001
3723323994840,5cyclictest3739629-21sshd12:24:300
3723323994835,10cyclictest3731815-21sshd09:54:000
3723323994835,10cyclictest3731815-21sshd09:53:590
3723323994834,11cyclictest3734422-21sshd10:43:240
3723323994829,15cyclictest363-21snmpd10:24:100
3723323994829,15cyclictest363-21snmpd10:24:100
3723323994828,15cyclictest363-21snmpd11:38:190
3723323994814,30cyclictest0-21swapper/007:40:020
3723323994814,30cyclictest0-21swapper/007:40:020
3723337994731,9cyclictest3738325-21sendmail12:00:003
3723337994731,12cyclictest272-21dbus-daemon08:05:013
3723333994730,14cyclictest3737256-21sshd11:37:222
3723333994712,6cyclictest30-21ksoftirqd/210:05:012
3723333994711,13cyclictest3728467-21systemd-run08:50:002
3723329994732,11cyclictest235-21systemd-journal12:34:351
372332399476,10cyclictest272-21dbus-daemon11:30:010
3723323994735,9cyclictest3735173-21sshd10:56:250
3723323994734,10cyclictest3740117-21sshd12:31:230
3723323994733,10cyclictest3732131-21munin-run10:00:010
3723323994729,14cyclictest363-21snmpd12:15:530
3723333994637,6cyclictest30-21ksoftirqd/211:05:032
3723333994632,11cyclictest3738834-21sshd12:09:232
3723333994632,10cyclictest3726618-21cron08:15:002
3723333994632,10cyclictest3726142-21munin-run08:05:012
3723333994631,11cyclictest363-21snmpd09:08:462
3723333994631,11cyclictest363-21snmpd09:08:452
3723333994615,26cyclictest3730322-21latency_hist09:25:022
3723323994638,5cyclictest3726852-21sshd08:16:240
3723333994534,9cyclictest30-21ksoftirqd/210:57:242
3723333994532,10cyclictest3728172-21sshd08:41:242
3723333994530,10cyclictest3738083-21munin-run11:55:002
3723323994538,4cyclictest3726603-21sshd08:12:230
3723323994536,6cyclictest3736542-21munin-run11:25:000
3723323994535,7cyclictest3723762-21sshd07:19:240
3723323994532,10cyclictest3739085-21sshd12:14:220
3723323994529,12cyclictest363-21snmpd11:43:360
3723323994528,13cyclictest363-21snmpd09:41:020
3723323994512,6cyclictest16850irq/134-800000010:17:240
3723337994436,5cyclictest37-21ksoftirqd/310:10:033
3723337994429,12cyclictest3729987-21sshd09:17:253
3723337994427,12cyclictest235-21systemd-journal08:45:013
3723337994427,12cyclictest235-21systemd-journal08:00:013
3723333994437,4cyclictest3732060-21kworker/u8:009:59:592
3723333994431,10cyclictest3734930-21sshd10:54:122
3723333994431,10cyclictest3724793-21sshd07:38:352
3723333994431,10cyclictest3724793-21sshd07:38:342
3723333994430,11cyclictest3727902-21sshd08:38:252
3723333994429,11cyclictest306-21in:imuxsock09:10:022
3723329994431,9cyclictest23-21ksoftirqd/111:54:591
3723329994431,10cyclictest235-21systemd-journal07:20:011
3723329994426,13cyclictest3734171-21cron10:40:001
3723329994426,12cyclictest235-21systemd-journal09:15:001
3723323994429,10cyclictest363-21snmpd11:57:370
3723323994427,13cyclictest363-21snmpd09:07:230
3723323994427,13cyclictest363-21snmpd09:07:220
3723323994426,14cyclictest3734664-21sshd10:45:250
3723323994426,14cyclictest3734664-21sshd10:45:240
3723323994424,16cyclictest3736290-21systemd-run11:20:010
372333799439,18cyclictest235-21systemd-journal12:40:003
3723337994334,5cyclictest3732368-21munin-plugin-st10:05:003
3723337994329,9cyclictest306-21in:imuxsock08:25:023
3723337994325,12cyclictest272-21dbus-daemon08:10:023
3723333994331,9cyclictest3739322-21sshd12:15:232
3723333994330,10cyclictest3740152-21munin-run12:35:002
3723333994329,10cyclictest3731028-21sshd09:38:242
3723333994329,10cyclictest3731028-21sshd09:38:242
3723333994329,10cyclictest3730802-21munin-run09:35:012
3723333994323,16cyclictest363-21snmpd10:43:412
3723333994322,16cyclictest3736534-21systemd-run11:25:002
3723323994336,4cyclictest3724286-21sshd07:28:240
3723323994325,14cyclictest3725830-21sshd07:59:240
3723323994323,14cyclictest0-21swapper/007:10:020
3723337994226,8cyclictest37-21ksoftirqd/309:40:013
3723337994226,8cyclictest37-21ksoftirqd/309:40:003
3723337994226,11cyclictest235-21systemd-journal10:50:013
3723337994226,11cyclictest235-21systemd-journal10:50:003
3723337994225,13cyclictest235-21systemd-journal09:05:013
3723337994219,16cyclictest3726890-21modprobe08:20:003
3723337994210,7cyclictest131rcu_preempt10:50:233
3723333994229,10cyclictest3738580-21sshd12:04:232
3723333994228,11cyclictest3731801-21sshd09:50:252
3723333994228,11cyclictest3731801-21sshd09:50:242
3723333994215,22cyclictest3737650-21sed11:45:022
372333399421,36cyclictest272-21dbus-daemon08:25:012
3723333994212,25cyclictest3731335-21munin-run09:45:002
3723329994227,10cyclictest235-21systemd-journal10:29:591
3723329994223,13cyclictest235-21systemd-journal08:30:001
3723323994229,10cyclictest12-21ksoftirqd/007:15:010
3723323994228,11cyclictest1-21systemd11:10:000
3723323994218,21cyclictest3730007-21cron09:20:010
372333799413,3cyclictest131rcu_preempt09:10:033
372333799413,3cyclictest131rcu_preempt09:10:023
3723337994131,6cyclictest37-21ksoftirqd/312:25:013
3723337994125,12cyclictest235-21systemd-journal11:45:003
3723337994124,12cyclictest272-21dbus-daemon07:50:013
3723337994123,14cyclictest235-21systemd-journal07:25:023
3723337994112,5cyclictest131rcu_preempt09:55:003
3723337994112,5cyclictest131rcu_preempt09:55:003
3723333994133,5cyclictest3725599-21munin-run07:55:002
3723333994130,8cyclictest3737798-21sshd11:49:222
3723333994129,9cyclictest3735966-21sshd11:14:152
3723333994128,9cyclictest363-21snmpd09:48:522
3723333994127,11cyclictest3736800-21wc11:30:022
3723333994123,14cyclictest363-21snmpd07:31:462
3723329994125,12cyclictest1-21systemd12:20:011
3723329994123,14cyclictest1-21systemd10:35:011
372333799405,4cyclictest131rcu_preempt07:45:023
3723333994029,8cyclictest3739622-21sshd12:23:232
3723333994026,11cyclictest3733094-21cron10:16:592
3723333994026,10cyclictest363-21snmpd07:23:342
3723333994024,12cyclictest363-21snmpd12:39:452
3723333994024,12cyclictest363-21snmpd10:08:562
3723333994022,14cyclictest363-21snmpd11:32:402
3723333994019,17cyclictest3726394-21unin-run08:10:012
3723333994019,16cyclictest3725082-21cron07:44:592
3723333994015,20cyclictest3733691-21sort10:30:002
372332999403,6cyclictest3740415-21readlink12:39:591
3723329994024,12cyclictest235-21systemd-journal09:20:001
3723323994026,10cyclictest3732859-21cron10:14:590
3723337993927,7cyclictest306-21in:imuxsock12:19:243
3723337993922,13cyclictest1-21systemd10:25:003
3723337993922,13cyclictest1-21systemd10:25:003
3723337993921,12cyclictest306-21in:imuxsock11:25:013
3723337993912,22cyclictest1-21systemd09:30:003
3723333993931,5cyclictest3730550-21munin-run09:30:002
3723333993927,9cyclictest363-21snmpd12:29:242
3723329993929,6cyclictest3735207-21sendmail11:00:011
3723329993926,10cyclictest23-21ksoftirqd/112:24:591
3723329993925,10cyclictest23-21ksoftirqd/108:20:021
3723329993923,12cyclictest23-21ksoftirqd/107:50:001
3723329993922,13cyclictest272-21dbus-daemon07:40:021
3723329993922,13cyclictest272-21dbus-daemon07:40:011
3723329993922,13cyclictest235-21systemd-journal12:05:021
3723329993912,8cyclictest3726054-21cat08:00:021
372333799388,21cyclictest235-21systemd-journal10:20:013
3723337993827,7cyclictest3728930-21sshd08:56:243
372333799382,3cyclictest131rcu_preempt12:10:013
3723337993819,15cyclictest37-21ksoftirqd/311:30:033
3723333993825,10cyclictest363-21snmpd10:49:052
3723333993825,10cyclictest363-21snmpd10:49:052
3723333993824,10cyclictest3724299-21cron07:30:012
3723329993830,6cyclictest23-21ksoftirqd/111:10:021
372332999382,5cyclictest1-21systemd11:15:001
3723329993821,13cyclictest23-21ksoftirqd/109:05:011
372332999381,3cyclictest131rcu_preempt10:45:001
3723323993819,14cyclictest3733647-21sshd10:29:240
372333799378,5cyclictest131rcu_preempt10:00:013
372333799376,6cyclictest131rcu_preempt10:40:033
3723337993726,7cyclictest3733884-21sshd10:31:243
3723337993726,7cyclictest3724489-21wc07:30:023
3723337993724,9cyclictest3728728-21wc08:55:013
3723337993724,9cyclictest3728728-21wc08:55:013
3723337993721,11cyclictest3735212-21run-parts11:00:013
3723333993725,9cyclictest30-21ksoftirqd/208:25:032
372332999376,9cyclictest23-21ksoftirqd/109:39:331
372332999376,9cyclictest23-21ksoftirqd/109:39:321
372332999375,11cyclictest0-21swapper/108:49:241
372332999374,23cyclictest221rcuc/110:20:011
372332999373,6cyclictest131rcu_preempt08:10:001
3723329993729,6cyclictest23-21ksoftirqd/111:05:011
3723329993725,9cyclictest3736261-21systemctl11:20:001
3723329993723,10cyclictest3737305-21tr11:39:591
3723329993723,10cyclictest3731571-21cron09:50:011
3723329993722,10cyclictest306-21in:imuxsock08:14:591
372332999372,10cyclictest3728220-21unin-run08:45:011
372332999371,3cyclictest131rcu_preempt11:35:021
372333799364,24cyclictest3738207-21cat11:55:023
3723337993626,6cyclictest3731856-21kworker/3:210:30:013
3723337993619,11cyclictest3739069-21kworker/u8:212:15:003
372333799361,3cyclictest131rcu_preempt12:05:033
372332999364,3cyclictest131rcu_preempt07:55:011
372332999362,8cyclictest0-21swapper/109:35:001
3723329993625,9cyclictest23-21ksoftirqd/112:29:311
3723329993624,9cyclictest23-21ksoftirqd/107:28:311
3723329993623,8cyclictest306-21in:imuxsock08:54:251
3723329993623,8cyclictest306-21in:imuxsock08:54:241
3723329993623,10cyclictest23-21ksoftirqd/109:57:241
372332999362,10cyclictest0-21swapper/108:35:031
3723329993620,11cyclictest1-21systemd11:45:001
372333799354,3cyclictest131rcu_preempt11:20:013
3723337993524,6cyclictest306-21in:imuxsock08:35:013
3723337993523,6cyclictest306-21in:imuxsock07:55:003
3723337993522,9cyclictest3723688-21sed07:15:023
3723337993521,10cyclictest235-21systemd-journal12:30:023
3723333993513,18cyclictest3723347-21latency_hist07:10:012
372332999353,23cyclictest3728131-21cat08:40:021
3723329993528,5cyclictest23-21ksoftirqd/108:05:021
3723329993526,6cyclictest3723483-21sshd07:10:241
372332999352,3cyclictest131rcu_preempt09:25:001
3723329993519,12cyclictest272-21dbus-daemon12:10:001
372333799346,22cyclictest3727157-21cron08:25:013
372333799345,4cyclictest131rcu_preempt07:40:013
372333799345,4cyclictest131rcu_preempt07:40:013
3723337993426,5cyclictest3730295-21munin-plugin-st09:25:003
372333799342,4cyclictest131rcu_preempt07:20:003
3723333993423,8cyclictest363-21snmpd11:55:462
3723329993427,4cyclictest3734959-21sh10:55:011
3723329993421,9cyclictest3738337-21sendmail12:00:001
3723329993421,9cyclictest23-21ksoftirqd/107:45:001
3723329993420,10cyclictest23-21ksoftirqd/110:21:251
3723329993420,10cyclictest23-21ksoftirqd/110:21:241
372333799333,22cyclictest3737273-21cron11:40:003
3723337993322,7cyclictest3734436-21cron10:45:003
3723337993321,9cyclictest3739968-21sed12:30:013
3723337993321,7cyclictest3737035-21munin-run11:35:003
3723337993317,12cyclictest0-21swapper/311:09:593
3723337993314,14cyclictest1-21systemd08:15:003
372332999335,4cyclictest131rcu_preempt10:15:001
372332999333,5cyclictest131rcu_preempt09:55:011
372332999333,5cyclictest131rcu_preempt09:55:001
3723329993325,5cyclictest3732606-21cron10:10:011
372332999332,4cyclictest131rcu_preempt10:50:011
372332999332,4cyclictest131rcu_preempt10:50:001
372332999331,8cyclictest3737815-21modprobe11:50:011
3723329993318,11cyclictest3729687-21cat09:10:021
3723329993318,11cyclictest3729687-21cat09:10:021
372333799324,3cyclictest131rcu_preempt11:50:003
3723337993222,7cyclictest3729751-21sh09:14:593
3723337993221,8cyclictest3731433-21cat09:45:013
3723337993219,8cyclictest306-21in:imuxsock08:50:013
372333799321,4cyclictest131rcu_preempt09:35:003
372332999322,4cyclictest131rcu_preempt11:30:001
3723329993224,6cyclictest23-21ksoftirqd/108:25:021
3723329993221,8cyclictest23-21ksoftirqd/109:45:001
3723337993110,17cyclictest3735976-21cron11:15:013
372332999314,4cyclictest131rcu_preempt09:30:021
372333799305,15cyclictest3727917-21modprobe08:40:003
372332999301,3cyclictest131rcu_preempt12:15:021
372333799294,3cyclictest131rcu_preempt10:15:003
372332999292,3cyclictest131rcu_preempt07:30:241
372332999283,18cyclictest3723332-21latency_hist07:10:011
3723329992810,14cyclictest272-21dbus-daemon07:25:011
372333799262,18cyclictest3723446-21grep07:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional