You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-03 - 02:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Thu Jul 02, 2026 12:45:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3723666997322,15cyclictest3811388-21ssh12:00:183
3723666997118,14cyclictest3784306-21ssh11:04:003
3723666997118,14cyclictest3784306-21ssh11:03:593
3723651997167,2cyclictest0-21swapper/008:30:010
3723666997022,14cyclictest3767608-21ssh10:29:053
3723666997017,18cyclictest3792700-21ssh11:21:163
3723666997016,21cyclictest3776941-21ssh10:48:413
3723659997027,10cyclictest1-21systemd11:15:011
3723659997027,10cyclictest1-21systemd11:15:001
3723666996920,14cyclictest3770728-21ssh10:35:193
3723666996821,15cyclictest3814071-21ssh12:06:053
372365199684,24cyclictest12-21ksoftirqd/011:20:000
372366299672,9cyclictest3794567-21ssh11:25:072
3723662996722,41cyclictest3774628-21sh10:43:532
3723666996617,18cyclictest3816241-21ssh12:10:323
372365999666,3cyclictest131rcu_preempt07:55:031
3723651996662,2cyclictest0-21swapper/010:05:010
372366699652,54cyclictest0-21swapper/307:55:033
3723666996517,16cyclictest3791955-21munin-run11:20:013
3723666996511,18cyclictest3778284-21ssh10:51:113
3723666996511,18cyclictest3778284-21ssh10:51:113
3723659996532,6cyclictest23-21ksoftirqd/111:30:021
3723666996417,14cyclictest3795150-21ssh11:26:233
372366299642,17cyclictest3724147-21munin-run07:20:022
3723666996326,12cyclictest3729828-21diskmemload11:12:003
3723666996326,12cyclictest3729828-21diskmemload11:12:003
3723666996323,14cyclictest3737289-21ssh09:25:133
3723666996317,15cyclictest3734461-21ssh09:19:373
3723666996311,17cyclictest3747782-21ssh09:47:323
3723662996352,6cyclictest3727784-21kworker/2:309:10:292
3723666996227,31cyclictest3725661-21cron07:50:003
3723666996224,13cyclictest3806665-21ssh11:50:183
3723662996230,28cyclictest30-21ksoftirqd/209:40:022
372366299622,20cyclictest3727759-21cut08:30:002
372366299622,18cyclictest3761923-21ssh10:17:062
372366299622,14cyclictest3726976-21munin-run08:15:012
3723662996220,38cyclictest30-21ksoftirqd/211:24:412
3723666996131,22cyclictest3729837-21sh09:10:003
3723666996121,13cyclictest3781648-21ssh10:58:303
3723666996114,13cyclictest3755550-21ssh10:03:483
3723662996125,31cyclictest30-21ksoftirqd/210:58:242
3723662996123,34cyclictest3785772-21sh11:06:552
372366299611,15cyclictest3724904-21munin-run07:35:002
3723666996044,12cyclictest3743123-21sshd09:37:443
3723666996044,12cyclictest3461527-1kworker/3:2H10:13:153
3723666996044,12cyclictest3461527-1kworker/3:2H10:13:143
3723666996042,14cyclictest3741441-21ssh09:34:103
3723666996013,17cyclictest3821520-21ssh12:21:373
3723662996026,30cyclictest3765135-21sh10:23:522
3723662996025,32cyclictest30-21ksoftirqd/209:45:012
372366299602,15cyclictest3754283-21ssh10:00:502
3723662996021,35cyclictest30-21ksoftirqd/212:14:312
3723662996021,34cyclictest3823283-21sh12:25:132
3723662996018,11cyclictest30-21ksoftirqd/211:39:482
3723662996017,5cyclictest3741612-21sh09:34:452
3723666995947,9cyclictest3731027-21ssh09:12:103
3723666995943,12cyclictest3799838-21ssh11:36:073
3723666995942,13cyclictest3751212-21ssh09:54:593
3723666995942,13cyclictest3751212-21ssh09:54:583
3723666995921,9cyclictest3797631-21ssh11:31:403
3723666995917,13cyclictest3825147-21rm12:29:293
3723666995917,12cyclictest0-21swapper/312:35:313
3723662995923,33cyclictest3820668-21cat12:20:022
3723662995922,33cyclictest30-21ksoftirqd/211:05:012
3723662995922,33cyclictest30-21ksoftirqd/211:05:012
372366299592,16cyclictest3797669-21ssh11:31:472
372366299592,14cyclictest3814137-21ssh12:06:152
3723662995916,9cyclictest3807802-1kworker/u9:011:53:182
3723662995915,8cyclictest3753765-21cut10:00:022
372366299591,15cyclictest3735074-21ssh09:20:222
3723659995929,9cyclictest23-21ksoftirqd/109:51:321
3723659995929,9cyclictest23-21ksoftirqd/109:51:321
3723659995924,11cyclictest275-21dbus-daemon08:15:011
3723651995940,16cyclictest16850irq/134-800000009:15:010
3723651995927,16cyclictest3742611-21ssh09:36:320
3723666995844,11cyclictest3735744-21ssh09:22:063
3723666995842,12cyclictest3769264-21ssh10:32:283
3723666995842,12cyclictest3764868-21ssh10:23:083
3723666995841,13cyclictest3761262-21ssh10:15:343
372366299582,15cyclictest3828174-21ssh12:35:362
372366299582,15cyclictest3826392-21ssh12:32:032
372366299582,13cyclictest3776141-21ssh10:46:452
3723662995813,7cyclictest3725911-21sshd07:54:442
372366299581,18cyclictest3750906-21ssh09:54:092
372366299581,18cyclictest3750906-21ssh09:54:082
372365199585,41cyclictest3467602-21systemd-journal09:45:010
3723651995853,3cyclictest0-21swapper/011:35:020
3723651995833,9cyclictest12-21ksoftirqd/011:55:020
3723651995829,12cyclictest12-21ksoftirqd/010:12:210
3723651995829,12cyclictest12-21ksoftirqd/010:12:200
3723651995820,18cyclictest16850irq/134-800000009:53:140
3723651995820,18cyclictest16850irq/134-800000009:53:140
372365199581,41cyclictest3817469-21ssh12:13:240
3723666995743,11cyclictest3818389-21ssh12:15:043
3723666995742,12cyclictest3801956-21ssh11:40:273
3723666995742,11cyclictest3752196-21ssh09:56:443
3723666995741,13cyclictest3757680-21ssh10:08:173
3723666995741,12cyclictest3809097-21ssh11:55:333
3723666995741,12cyclictest3804310-21ssh11:45:273
3723666995740,14cyclictest3745296-21ssh09:42:063
3723666995739,15cyclictest3785526-21ssh11:06:213
372366299572,16cyclictest3726727-21unin-run08:10:012
372366299572,14cyclictest3813414-21munin-run12:05:012
372366299572,13cyclictest3767958-21ssh10:30:002
3723662995714,7cyclictest3728441-21sed08:40:032
3723662995714,7cyclictest3728441-21sed08:40:022
372366299571,15cyclictest3778620-21ssh10:52:022
372366299571,15cyclictest3778620-21ssh10:52:022
372366299571,13cyclictest3467602-21systemd-journal08:20:012
3723651995726,15cyclictest3783566-21ssh11:02:170
3723651995726,15cyclictest3783566-21ssh11:02:170
3723651995719,18cyclictest16850irq/134-800000009:24:120
372365199571,40cyclictest3809783-21ssh11:57:100
3723666995640,13cyclictest3826190-21ssh12:31:273
3723666995640,13cyclictest3773014-21ssh10:40:053
3723666995636,15cyclictest3726461-21munin-run08:05:013
3723666995635,17cyclictest3726949-21cron08:15:013
372366299563,18cyclictest3729841-21ssh09:10:002
3723662995627,26cyclictest30-21ksoftirqd/209:29:362
3723662995623,29cyclictest30-21ksoftirqd/209:50:012
372366299562,14cyclictest3809802-21ssh11:57:132
3723651995635,17cyclictest3772887-21ssh10:40:010
3723651995630,11cyclictest3773548-21ssh10:41:200
3723651995628,12cyclictest3822413-21ssh12:23:480
3723651995627,25cyclictest3803526-21ssh11:44:170
3723651995618,7cyclictest16850irq/134-800000010:48:500
3723651995618,18cyclictest16850irq/134-800000012:38:400
3723651995618,18cyclictest16850irq/134-800000010:28:170
3723651995618,18cyclictest16850irq/134-800000009:31:120
3723651995617,19cyclictest16850irq/134-800000011:36:530
3723666995540,11cyclictest3726695-21sshd08:09:433
3723666995540,11cyclictest3724639-21munin-run07:30:013
3723666995540,11cyclictest3724639-21munin-run07:30:003
3723662995533,15cyclictest275-21dbus-daemon11:45:012
3723662995527,24cyclictest3790441-21sh11:16:422
372366299552,13cyclictest3723862-21munin-run07:15:002
3723662995519,8cyclictest30-21ksoftirqd/211:46:272
3723662995514,36cyclictest0-21swapper/208:35:022
372366299551,15cyclictest3769201-21sh10:32:192
3723651995544,6cyclictest3820585-21sendmail12:20:010
3723651995527,12cyclictest3789205-21ssh11:14:240
3723651995527,12cyclictest3789205-21ssh11:14:240
3723651995518,6cyclictest16850irq/134-800000009:17:280
3723651995518,17cyclictest16850irq/134-800000012:31:000
3723651995518,17cyclictest16850irq/134-800000011:06:030
3723651995518,17cyclictest16850irq/134-800000010:55:060
3723651995518,17cyclictest16850irq/134-800000010:16:260
3723651995517,5cyclictest16850irq/134-800000009:46:170
3723651995517,18cyclictest16850irq/134-800000011:46:050
3723651995517,18cyclictest16850irq/134-800000010:06:060
3723651995517,17cyclictest16850irq/134-800000012:01:320
3723651995516,18cyclictest16850irq/134-800000010:21:150
3723666995440,10cyclictest3728541-21munin-run08:45:003
3723666995431,17cyclictest3729281-21cron09:00:003
3723662995446,5cyclictest3784738-21kworker/2:011:11:492
3723662995446,5cyclictest3784738-21kworker/2:011:11:482
372366299543,16cyclictest3734760-21latency_hist09:20:022
372366299542,14cyclictest3467602-21systemd-journal08:22:452
372366299542,14cyclictest3467602-21systemd-journal08:22:442
372366299542,13cyclictest3729674-21latency_hist09:05:022
372366299541,16cyclictest3771332-21ssh10:36:432
3723651995418,6cyclictest16850irq/134-800000010:51:340
3723651995418,6cyclictest16850irq/134-800000010:51:330
3723651995417,6cyclictest16850irq/134-800000012:26:120
3723651995417,6cyclictest16850irq/134-800000010:34:320
3723651995417,17cyclictest16850irq/134-800000012:06:340
3723651995416,6cyclictest16850irq/134-800000009:26:370
3723651995416,18cyclictest16850irq/134-800000009:58:220
3723651995415,18cyclictest16850irq/134-800000011:30:000
3723651995415,18cyclictest16850irq/134-800000011:20:170
3723666995339,11cyclictest3723818-21sshd07:11:443
3723666995338,11cyclictest3727183-21sshd08:15:443
3723666995335,14cyclictest48-21kcompactd007:35:023
3723662995317,8cyclictest30-21ksoftirqd/207:23:322
3723662995316,14cyclictest275-21dbus-daemon10:10:012
3723666995241,8cyclictest364-21snmpd07:17:423
3723666995238,11cyclictest3729542-21sshd09:00:433
3723666995238,11cyclictest3728761-21sshd08:46:453
3723662995243,5cyclictest3727784-21kworker/2:308:59:592
372366299522,13cyclictest3726467-21true08:05:002
3723662995215,32cyclictest3467602-21systemd-journal08:00:012
3723666995139,9cyclictest3727468-21sshd08:20:253
3723666995139,9cyclictest3727468-21sshd08:20:253
3723666995138,9cyclictest48-21kcompactd008:53:503
372366299512,12cyclictest1-21systemd07:50:002
3723662995119,28cyclictest3729828-21diskmemload10:10:352
3723662995119,28cyclictest3729828-21diskmemload10:10:342
372366299511,15cyclictest3729019-21sshd08:52:442
372366299511,14cyclictest3821198-21ssh12:20:572
372365199512,28cyclictest0-21swapper/007:34:190
3723662995011,5cyclictest3724615-21sshd07:27:452
3723662995011,5cyclictest3724615-21sshd07:27:442
3723651995042,4cyclictest16950irq/224-800000009:00:010
372365199504,10cyclictest3725141-21modprobe07:39:590
3723666994936,10cyclictest3725438-21munin-run07:45:023
3723666994936,10cyclictest364-21snmpd07:50:493
3723666994935,10cyclictest3724386-21munin-run07:25:013
3723666994932,12cyclictest3723803-21sed07:10:023
3723662994916,11cyclictest3725160-21tr07:40:012
3723659994936,10cyclictest3726708-21cron08:09:591
3723666994839,6cyclictest3727757-21munin-run08:29:593
3723666994830,13cyclictest3467602-21systemd-journal08:32:453
3723659994835,10cyclictest23-21ksoftirqd/110:23:431
3723651994833,12cyclictest3467602-21systemd-journal07:20:030
3723666994734,9cyclictest364-21snmpd07:33:433
372365999478,6cyclictest131rcu_preempt08:20:001
3723659994731,11cyclictest275-21dbus-daemon09:50:001
3723659994712,6cyclictest131rcu_preempt12:15:091
3723651994733,9cyclictest3729826-21cron09:10:000
3723662994612,7cyclictest3723785-21date07:10:012
3723659994634,9cyclictest3797221-21sshd11:30:441
3723659994629,12cyclictest3825369-21systemd-run12:30:001
372365999461,21cyclictest0-21swapper/111:46:401
3723662994512,5cyclictest3725484-21latency_hist07:45:022
372365999457,11cyclictest3801486-21cron11:40:011
3723659994535,6cyclictest309-21in:imuxsock11:25:001
3723659994511,5cyclictest131rcu_preempt09:13:181
372365199458,33cyclictest3727226-21latency_hist08:20:010
3723651994536,3cyclictest150-21pr/ttyS209:05:020
3723651994528,13cyclictest3725652-21sshd07:47:440
3723651994527,14cyclictest3467602-21systemd-journal08:05:000
3723666994431,10cyclictest364-21snmpd08:35:243
3723666994431,10cyclictest364-21snmpd08:35:233
3723662994412,9cyclictest3467602-21systemd-journal08:45:002
372365999446,6cyclictest23-21ksoftirqd/110:12:331
372365999446,6cyclictest23-21ksoftirqd/110:12:331
3723659994430,10cyclictest3784185-21sshd11:03:451
3723659994430,10cyclictest3784185-21sshd11:03:451
3723659994430,10cyclictest309-21in:imuxsock09:55:031
3723659994425,13cyclictest275-21dbus-daemon09:00:011
3723659994423,14cyclictest3467602-21systemd-journal08:45:001
3723659994411,5cyclictest131rcu_preempt10:58:471
3723651994433,6cyclictest19350irq/34-mmc108:45:010
372365999433,9cyclictest0-21swapper/109:26:101
3723659994327,12cyclictest23-21ksoftirqd/110:32:551
3723659994326,12cyclictest3467602-21systemd-journal08:30:001
372365999432,3cyclictest131rcu_preempt11:20:001
372366299422,14cyclictest3728929-21cat08:50:022
3723659994231,5cyclictest274-21cron08:40:011
3723659994231,5cyclictest274-21cron08:40:001
3723659994229,9cyclictest23-21ksoftirqd/110:19:591
3723651994235,4cyclictest16950irq/224-800000008:00:020
3723651994231,8cyclictest12-21ksoftirqd/008:10:020
372365199422,7cyclictest3724232-21cat07:20:030
3723651994224,11cyclictest12-21ksoftirqd/008:25:010
3723651994224,11cyclictest12-21ksoftirqd/008:25:000
372365199422,35cyclictest3728020-21unin-run08:35:010
372365999418,4cyclictest131rcu_preempt11:43:051
3723659994132,5cyclictest3729827-21latency_hist09:09:591
3723659994131,7cyclictest23-21ksoftirqd/110:53:431
3723659994131,7cyclictest23-21ksoftirqd/110:53:421
3723659994130,9cyclictest23-21ksoftirqd/109:05:021
3723659994130,8cyclictest3467602-21systemd-journal07:20:431
3723659994128,8cyclictest3813550-21cat12:05:021
3723659994126,12cyclictest1-21systemd10:50:011
372365999411,5cyclictest131rcu_preempt12:15:001
3723659994030,7cyclictest3772832-21ssh10:40:021
3723659994027,9cyclictest3467602-21systemd-journal08:50:001
3723659994026,8cyclictest0-21swapper/112:30:381
372365999402,5cyclictest131rcu_preempt09:40:031
372365999402,4cyclictest0-21swapper/109:22:311
3723659994020,15cyclictest3467602-21systemd-journal07:35:001
372365199404,10cyclictest12-21ksoftirqd/008:40:010
372365199404,10cyclictest12-21ksoftirqd/008:40:000
372365999395,3cyclictest131rcu_preempt10:05:011
372365999394,5cyclictest131rcu_preempt12:35:321
3723659993926,10cyclictest23-21ksoftirqd/112:00:011
3723659993926,10cyclictest23-21ksoftirqd/110:09:061
372365999392,3cyclictest131rcu_preempt08:05:021
3723659993910,5cyclictest131rcu_preempt10:28:431
372365199393,4cyclictest131rcu_preempt07:15:000
372365199392,33cyclictest3729021-21sshd08:52:440
372365999384,3cyclictest131rcu_preempt09:40:031
372365999382,4cyclictest131rcu_preempt11:06:431
372365999382,3cyclictest131rcu_preempt12:25:001
3723659993821,12cyclictest275-21dbus-daemon07:15:011
372365199387,5cyclictest131rcu_preempt07:26:190
372365199387,5cyclictest131rcu_preempt07:26:180
3723651993827,8cyclictest12-21ksoftirqd/007:53:020
372365999379,6cyclictest131rcu_preempt11:52:361
372365999373,3cyclictest131rcu_preempt12:05:421
3723659993727,6cyclictest309-21in:imuxsock10:40:021
3723659993725,8cyclictest23-21ksoftirqd/109:15:231
3723651993726,7cyclictest3726961-21munin-plugin-st08:15:010
372365999363,4cyclictest131rcu_preempt09:33:091
3723659993622,10cyclictest3724785-21sed07:30:031
3723659993622,10cyclictest3724785-21sed07:30:021
3723659993621,12cyclictest23-21ksoftirqd/108:52:431
372365999361,5cyclictest131rcu_preempt07:10:011
3723651993622,10cyclictest1-21systemd08:50:000
372365999352,3cyclictest131rcu_preempt08:25:011
372365999352,3cyclictest131rcu_preempt08:25:001
372365999352,3cyclictest131rcu_preempt07:20:001
3723659993523,9cyclictest23-21ksoftirqd/107:51:201
3723659993520,11cyclictest23-21ksoftirqd/107:41:431
372365199354,22cyclictest311-21rs:main0
3723651993517,16cyclictest3723645-21cyclictest07:10:020
372365999341,4cyclictest131rcu_preempt08:33:441
3723659993225,4cyclictest3725823-21latency_hist07:50:021
372365999321,3cyclictest131rcu_preempt07:40:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional