You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-11 - 18:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Sun Jan 11, 2026 12:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4011274999590,3cyclictest0-21swapper/010:10:020
4011274997729,15cyclictest131rcu_preempt11:11:270
4011274997129,16cyclictest131rcu_preempt09:15:000
4011274997129,16cyclictest131rcu_preempt09:15:000
4011274997113,50cyclictest12-21ksoftirqd/012:28:480
4011274996948,7cyclictest12-21ksoftirqd/012:37:180
4011288996727,36cyclictest3467602-21systemd-journal09:12:343
4011288996727,36cyclictest3467602-21systemd-journal09:12:343
4011274996636,9cyclictest12-21ksoftirqd/010:20:020
4011288996526,35cyclictest4020934-21sh09:17:213
4011288996517,12cyclictest1-21systemd12:25:013
4011288996517,12cyclictest1-21systemd12:25:013
401127499658,53cyclictest12-21ksoftirqd/010:24:180
4011274996523,37cyclictest12-21ksoftirqd/010:57:440
4011288996425,34cyclictest4017437-21diskmemload10:20:043
4011288996327,33cyclictest37-21ksoftirqd/311:18:313
4011288996325,5cyclictest4059492-21sh10:40:443
4011288996320,13cyclictest4068145-21basename11:00:003
4011288996319,5cyclictest4099441-21sh12:07:563
4011288996126,30cyclictest4015619-21latency_hist08:35:023
4011288996125,6cyclictest4037029-21sh09:52:223
4011288996125,32cyclictest4103982-21sh12:17:423
401128899612,20cyclictest4065851-21cut10:55:003
401128899612,20cyclictest4065851-21cut10:55:003
4011288996119,12cyclictest0-21swapper/312:40:013
4011288996118,11cyclictest37-21ksoftirqd/311:02:043
4011288996117,7cyclictest4054553-21sshd10:30:073
4011274996138,17cyclictest4098047-21cron12:05:000
4011274996136,17cyclictest12-21ksoftirqd/012:18:380
4011274996115,42cyclictest12-21ksoftirqd/009:17:200
4011274996112,40cyclictest12-21ksoftirqd/011:08:150
4011288996043,13cyclictest3467602-21systemd-journal08:55:013
4011288996025,6cyclictest4090670-21sh11:49:003
4011288996017,8cyclictest4026466-21sshd09:29:343
4011288996017,7cyclictest4014843-21cron08:20:003
4011288995925,6cyclictest4027109-21sh09:30:433
4011288995925,12cyclictest37-21ksoftirqd/307:20:023
4011288995925,12cyclictest37-21ksoftirqd/307:20:023
401128899592,19cyclictest4047405-21unin-run10:15:013
4011274995955,2cyclictest0-21swapper/008:00:020
401127499593,50cyclictest4086575-21grep11:40:000
401127499593,50cyclictest4086575-21grep11:40:000
401127499591,5cyclictest131rcu_preempt11:45:010
4011274995914,41cyclictest12-21ksoftirqd/009:24:280
4011288995842,12cyclictest3467602-21systemd-journal08:45:013
4011288995842,12cyclictest3467602-21systemd-journal08:45:013
4011288995837,15cyclictest3467602-21systemd-journal12:35:003
4011288995826,28cyclictest4012263-21/usr/sbin/munin07:30:013
4011288995823,4cyclictest4017437-21diskmemload11:08:503
4011288995820,5cyclictest4017437-21diskmemload10:25:583
4011288995820,5cyclictest4017437-21diskmemload10:25:573
4011288995725,7cyclictest37-21ksoftirqd/312:03:193
4011288995722,5cyclictest4108060-21sh12:26:233
4011288995717,13cyclictest309-21in:imuxsock10:10:013
401128899571,19cyclictest4012971-21cat07:40:023
401127499577,7cyclictest12-21ksoftirqd/009:50:010
4011274995738,13cyclictest1-21systemd10:05:010
401127499573,14cyclictest4031242-21latency_hist09:40:000
4011274995729,5cyclictest4063613-21latency_hist10:50:020
4011288995625,4cyclictest4085314-21sh11:37:033
4011288995625,4cyclictest4085314-21sh11:37:033
4011288995620,4cyclictest4017437-21diskmemload11:56:153
4011288995614,10cyclictest4081976-21cron11:30:003
401127499563,14cyclictest4079933-21latency_hist11:25:020
401127499561,38cyclictest131rcu_preempt12:30:190
4011288995520,9cyclictest37-21ksoftirqd/311:24:463
4011288995518,5cyclictest4084296-21cron11:35:003
401127499557,38cyclictest16950irq/224-800000011:35:010
4011274995526,14cyclictest4056094-21ssh10:33:440
4011288995436,13cyclictest3467602-21systemd-journal11:55:003
401128899543,13cyclictest4033557-21modprobe09:45:003
4011288995421,4cyclictest4014577-21latency_hist08:15:013
4011274995419,31cyclictest0-21swapper/009:50:150
4011274995418,4cyclictest40284721ssh09:33:550
4011274995416,14cyclictest131rcu_preempt11:58:170
4011288995320,5cyclictest4074440-21rm11:13:253
4011288995318,9cyclictest37-21ksoftirqd/310:04:583
4011274995338,11cyclictest3467602-21systemd-journal08:35:000
401127499533,45cyclictest4065981-21ssh10:55:020
401127499533,45cyclictest4065981-21ssh10:55:020
4011274995334,13cyclictest3467602-21systemd-journal08:50:000
401127499532,14cyclictest275-21dbus-daemon10:30:000
401127499532,14cyclictest275-21dbus-daemon10:30:000
4011288995225,6cyclictest37-21ksoftirqd/309:56:153
4011288995225,6cyclictest37-21ksoftirqd/309:56:153
4011288995221,6cyclictest37-21ksoftirqd/309:20:253
4011288995221,5cyclictest37-21ksoftirqd/309:46:273
4011288995217,8cyclictest37-21ksoftirqd/310:47:203
4011288995216,12cyclictest1-21systemd12:15:013
4011288995216,12cyclictest1-21systemd12:15:013
401127499523,44cyclictest4058935-21modprobe10:40:000
401128899512,21cyclictest4012011-21munin-run07:24:593
4011288995116,8cyclictest37-21ksoftirqd/308:40:033
4011274995131,14cyclictest275-21dbus-daemon10:45:010
401127499512,9cyclictest3467602-21systemd-journal09:00:000
4011274995122,25cyclictest12-21ksoftirqd/011:29:470
401127499511,3cyclictest0-21swapper/008:10:020
401127499511,3cyclictest0-21swapper/008:10:020
4011280995029,15cyclictest275-21dbus-daemon10:55:011
4011280995029,15cyclictest275-21dbus-daemon10:55:011
401128099501,33cyclictest4088953-21ssh11:45:011
4011274995025,19cyclictest4047396-21ssh10:15:010
4011274995020,3cyclictest12-21ksoftirqd/009:56:440
4011274995020,3cyclictest12-21ksoftirqd/009:56:440
4011288994923,7cyclictest37-21ksoftirqd/310:39:313
4011274994938,8cyclictest19350irq/34-mmc112:10:010
401127499492,18cyclictest4070408-21cron11:05:000
4011284994834,11cyclictest30-21ksoftirqd/211:25:002
401127499484,13cyclictest1-21systemd11:55:010
401127499483,8cyclictest4015882-21systemctl08:40:000
4011274994830,13cyclictest4017437-21diskmemload12:25:010
4011274994830,13cyclictest4017437-21diskmemload12:25:010
401127499482,41cyclictest4017437-21diskmemload09:10:000
4011274994821,11cyclictest4026306-21ssh09:29:110
4011284994731,11cyclictest275-21dbus-daemon10:25:012
4011284994729,14cyclictest1-21systemd08:35:012
4011280994721,9cyclictest23-21ksoftirqd/107:25:011
4011274994735,7cyclictest4011276-21cat07:10:010
4011274994732,12cyclictest3467602-21systemd-journal12:10:020
4011274994732,12cyclictest3467602-21systemd-journal12:10:020
4011274994729,14cyclictest1-21systemd08:30:000
4011274994715,28cyclictest12-21ksoftirqd/011:17:160
4011274994714,8cyclictest12-21ksoftirqd/009:42:180
401128499464,3cyclictest131rcu_preempt10:12:382
4011284994629,13cyclictest1-21systemd09:35:012
401128099463,4cyclictest131rcu_preempt10:10:401
4011280994631,12cyclictest4052248-21sshd10:25:061
4011280994631,12cyclictest4052248-21sshd10:25:061
4011280994629,13cyclictest275-21dbus-daemon08:50:001
4011284994524,16cyclictest275-21dbus-daemon08:40:012
401128499452,3cyclictest131rcu_preempt11:51:402
401128499451,4cyclictest131rcu_preempt10:35:222
401128099453,32cyclictest4020736-21kworker/u8:009:20:021
4011280994524,15cyclictest4095785-21ssh12:00:011
401128099451,4cyclictest131rcu_preempt10:21:391
401128099451,3cyclictest131rcu_preempt08:40:011
4011274994528,13cyclictest4016622-21sshd08:51:340
401128899447,3cyclictest131rcu_preempt10:20:023
4011288994433,6cyclictest4016893-21cron09:00:003
4011288994430,10cyclictest4013807-21systemctl08:00:003
4011284994431,10cyclictest4024349-21pluginstate09:25:002
4011284994426,14cyclictest4022009-21run-parts09:20:002
4011280994430,11cyclictest4105158-21wc12:20:021
4011280994427,13cyclictest3467602-21systemd-journal12:15:011
4011280994427,13cyclictest3467602-21systemd-journal12:15:011
401128099442,28cyclictest4100463-21ssh12:10:021
401127499444,30cyclictest4016152-21munin-run08:45:010
401127499444,30cyclictest4016152-21munin-run08:45:010
401127499443,34cyclictest3467602-21systemd-journal07:50:010
401127499442,19cyclictest131rcu_preempt11:50:010
4011288994326,13cyclictest3467602-21systemd-journal08:00:023
4011288994325,14cyclictest3467602-21systemd-journal11:45:003
4011284994330,9cyclictest275-21dbus-daemon10:35:002
4011284994329,10cyclictest4031285-21pluginstate09:40:022
4011284994320,14cyclictest30-21ksoftirqd/208:15:022
401128499431,3cyclictest131rcu_preempt12:36:172
401128499431,3cyclictest131rcu_preempt12:34:282
401128499431,3cyclictest131rcu_preempt12:29:432
401128499431,3cyclictest131rcu_preempt10:50:462
401128499431,3cyclictest131rcu_preempt10:50:462
401128099437,23cyclictest4017437-21diskmemload09:55:021
4011280994332,7cyclictest4093506-21cat11:55:021
4011280994327,13cyclictest3467602-21systemd-journal10:10:011
4011280994319,20cyclictest275-21dbus-daemon10:05:001
401128099431,4cyclictest0-21swapper/112:38:181
401128099431,3cyclictest4091133-21ssh11:50:011
4011280994311,16cyclictest23-21ksoftirqd/110:57:511
4011274994339,2cyclictest0-21swapper/007:30:020
4011274994332,7cyclictest4011734-21sendmail07:20:010
4011274994332,7cyclictest4011734-21sendmail07:20:000
4011274994330,8cyclictest4011490-21latency_hist07:15:010
4011274994327,12cyclictest3467602-21systemd-journal07:44:590
4011288994226,12cyclictest3467602-21systemd-journal07:11:343
4011284994232,7cyclictest4036303-21sshd09:50:372
4011284994231,7cyclictest3467602-21systemd-journal07:20:012
4011284994231,7cyclictest3467602-21systemd-journal07:20:012
4011284994228,10cyclictest4017903-21sshd09:10:342
4011284994228,10cyclictest4017903-21sshd09:10:342
4011284994227,11cyclictest4098147-21date12:05:022
4011284994227,10cyclictest4013281-21cron07:50:002
401128499422,3cyclictest131rcu_preempt10:58:042
401128499422,34cyclictest4015359-21cron08:30:002
4011284994220,16cyclictest1-21systemd09:10:012
401128499421,6cyclictest0-21swapper/210:48:262
4011280994233,6cyclictest4014846-21sendmail08:20:001
4011280994230,8cyclictest4077426-21cron11:20:011
4011280994229,9cyclictest4016914-21systemctl09:00:001
4011280994227,10cyclictest3467602-21systemd-journal09:40:001
4011280994226,12cyclictest275-21dbus-daemon07:45:011
4011280994223,6cyclictest23-21ksoftirqd/107:35:011
4011280994223,6cyclictest23-21ksoftirqd/107:35:011
401128499414,3cyclictest131rcu_preempt11:37:172
401128499414,3cyclictest131rcu_preempt11:37:172
401128499413,3cyclictest131rcu_preempt10:27:032
401128499413,3cyclictest131rcu_preempt10:27:032
4011284994125,13cyclictest3467602-21systemd-journal09:42:342
401128499412,4cyclictest131rcu_preempt10:08:252
401128099418,18cyclictest23-21ksoftirqd/111:28:491
401128099413,3cyclictest131rcu_preempt09:21:051
4011280994126,10cyclictest3467602-21systemd-journal09:57:371
4011280994126,10cyclictest3467602-21systemd-journal09:57:371
401128099411,6cyclictest0-21swapper/109:41:191
401128099411,4cyclictest131rcu_preempt11:12:261
401128099411,3cyclictest131rcu_preempt09:14:311
401128099411,3cyclictest131rcu_preempt09:14:311
401127499412,7cyclictest4014056-21sshd08:04:340
401127499412,4cyclictest131rcu_preempt08:17:340
4011274994110,6cyclictest4012036-21kworker/0:007:31:470
4011274994110,6cyclictest4012036-21kworker/0:007:31:470
4011288994018,16cyclictest3467602-21systemd-journal08:10:013
4011288994018,16cyclictest3467602-21systemd-journal08:10:013
4011284994031,5cyclictest309-21in:imuxsock08:55:022
4011284994030,7cyclictest4088875-21pluginstate11:45:012
4011284994028,9cyclictest4016650-21cron08:55:002
4011284994026,10cyclictest30-21ksoftirqd/211:11:552
4011284994025,10cyclictest3467602-21systemd-journal07:35:012
4011284994025,10cyclictest3467602-21systemd-journal07:35:002
401128499402,3cyclictest131rcu_preempt11:58:312
401128499401,3cyclictest131rcu_preempt10:44:452
4011280994030,7cyclictest4013778-21sshd07:59:341
4011280994019,16cyclictest275-21dbus-daemon08:45:011
4011280994019,16cyclictest275-21dbus-daemon08:45:011
401128099401,8cyclictest131rcu_preempt11:05:471
4011280994016,4cyclictest23-21ksoftirqd/112:35:011
4011274994036,2cyclictest0-21swapper/008:10:020
4011288993930,6cyclictest4015370-21munin-plugin-st08:30:003
401128499394,4cyclictest131rcu_preempt12:14:392
401128499394,4cyclictest131rcu_preempt12:14:392
4011284993926,9cyclictest30-21ksoftirqd/209:50:022
401128499392,3cyclictest131rcu_preempt09:25:102
4011284993923,7cyclictest30-21ksoftirqd/207:36:342
401128499391,34cyclictest3467602-21systemd-journal11:50:022
401128099398,16cyclictest23-21ksoftirqd/111:01:481
401128099398,16cyclictest23-21ksoftirqd/110:39:421
401128099393,4cyclictest131rcu_preempt09:25:351
4011280993930,6cyclictest4107448-21sed12:25:021
4011280993930,6cyclictest4107448-21sed12:25:021
4011280993928,9cyclictest23-21ksoftirqd/110:20:011
4011280993928,8cyclictest4014757-21cat08:15:021
4011280993926,11cyclictest23-21ksoftirqd/110:44:581
4011280993926,10cyclictest23-21ksoftirqd/110:47:391
4011280993924,10cyclictest275-21dbus-daemon08:30:001
401128099392,3cyclictest131rcu_preempt07:20:021
401128099392,3cyclictest131rcu_preempt07:20:021
4011280993919,16cyclictest275-21dbus-daemon07:55:011
401127499397,5cyclictest131rcu_preempt08:25:010
401127499397,5cyclictest131rcu_preempt08:25:010
4011288993829,7cyclictest37-21ksoftirqd/309:04:343
4011288993827,8cyclictest4017462-21pluginstate09:10:013
4011288993811,6cyclictest131rcu_preempt09:40:013
4011284993829,6cyclictest0-21swapper/207:10:002
4011284993825,9cyclictest4013802-21cat07:59:592
4011284993825,9cyclictest309-21in:imuxsock08:01:352
401128499382,3cyclictest131rcu_preempt11:29:392
401128499381,6cyclictest0-21swapper/211:06:552
401128499381,3cyclictest131rcu_preempt12:10:002
401128099387,16cyclictest23-21ksoftirqd/111:33:041
4011280993829,5cyclictest4079726-21sh11:25:001
4011280993827,9cyclictest23-21ksoftirqd/112:05:031
4011280993825,10cyclictest4085604-21sshd11:37:451
4011280993825,10cyclictest4085604-21sshd11:37:451
401128099381,20cyclictest0-21swapper/109:31:551
4011274993816,14cyclictest275-21dbus-daemon07:40:010
401128499374,4cyclictest131rcu_preempt11:19:502
4011284993727,7cyclictest3467602-21systemd-journal10:00:012
4011284993727,7cyclictest3467602-21systemd-journal10:00:012
4011284993725,10cyclictest30-21ksoftirqd/210:18:432
4011284993724,11cyclictest30-21ksoftirqd/208:45:022
4011284993724,11cyclictest30-21ksoftirqd/208:45:022
4011284993721,12cyclictest0-21swapper/212:20:002
4011284993720,12cyclictest1-21systemd09:05:002
401128099373,6cyclictest0-21swapper/109:47:161
4011280993727,8cyclictest23-21ksoftirqd/112:27:151
4011280993723,11cyclictest23-21ksoftirqd/110:35:021
4011280993721,11cyclictest3467602-21systemd-journal08:35:001
4011280993718,14cyclictest275-21dbus-daemon08:25:011
4011280993718,14cyclictest275-21dbus-daemon08:25:011
4011274993720,12cyclictest3467602-21systemd-journal07:55:020
4011288993626,7cyclictest4013603-21latency_hist07:55:023
4011284993628,6cyclictest30-21ksoftirqd/212:24:022
4011284993628,6cyclictest30-21ksoftirqd/212:24:012
4011284993628,6cyclictest30-21ksoftirqd/211:00:162
4011284993625,8cyclictest4084321-21pluginstate11:35:012
401127499361,5cyclictest131rcu_preempt09:05:010
4011288993526,5cyclictest4015108-21cron08:25:013
4011288993526,5cyclictest4015108-21cron08:25:003
4011284993525,8cyclictest30-21ksoftirqd/210:02:522
4011284993524,9cyclictest30-21ksoftirqd/207:15:002
401128499351,3cyclictest131rcu_preempt08:25:012
401128499351,3cyclictest131rcu_preempt08:25:002
4011280993525,7cyclictest1-21systemd08:55:011
4011280993521,11cyclictest275-21dbus-daemon09:05:011
4011288993426,5cyclictest4013291-21latency_hist07:50:023
401128499342,3cyclictest131rcu_preempt07:30:012
4011284993421,10cyclictest30-21ksoftirqd/207:25:022
4011284993417,5cyclictest131rcu_preempt08:17:342
4011280993427,4cyclictest4013291-21latency_hist07:50:021
4011280993424,6cyclictest23-21ksoftirqd/107:15:011
4011280993423,9cyclictest23-21ksoftirqd/107:30:021
401127499341,5cyclictest131rcu_preempt07:22:340
4011288993325,5cyclictest0-21swapper/307:10:013
4011288993322,8cyclictest37-21ksoftirqd/308:49:483
401128099333,3cyclictest131rcu_preempt09:10:011
4011280993321,9cyclictest23-21ksoftirqd/108:05:011
4011280993316,13cyclictest23-21ksoftirqd/107:39:591
401128499323,4cyclictest131rcu_preempt07:55:022
4011280993224,6cyclictest23-21ksoftirqd/107:10:021
4011280993222,8cyclictest23-21ksoftirqd/108:10:011
4011280993222,8cyclictest23-21ksoftirqd/108:10:011
4011288993113,13cyclictest275-21dbus-daemon07:35:013
4011288993113,13cyclictest275-21dbus-daemon07:35:013
4011284993123,5cyclictest4013237-21latency_hist07:45:022
4011284992922,5cyclictest4016404-21/usr/sbin/munin08:50:002
401128899283,3cyclictest131rcu_preempt07:45:003
4011284992712,13cyclictest4014294-21sshd08:05:342
4011284992712,13cyclictest4014294-21sshd08:05:342
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional