You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 22:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Tue Aug 25, 2026 12:45:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
360451998716,32cyclictest0-21swapper/208:25:012
360451998216,36cyclictest363242-21cut08:00:022
36045199812,42cyclictest455304-21ssh12:16:052
360444997974,2cyclictest0-21swapper/010:40:020
360451997824,21cyclictest30-21ksoftirqd/211:09:482
360451997824,21cyclictest30-21ksoftirqd/211:09:472
360444997872,3cyclictest0-21swapper/007:25:020
36045199732,41cyclictest410700-21ssh10:42:272
36045199722,37cyclictest391746-21ssh10:02:292
360451997221,24cyclictest30-21ksoftirqd/211:16:472
360451997221,24cyclictest30-21ksoftirqd/211:16:472
36044499725,42cyclictest19350irq/34-mmc109:55:020
360444997129,13cyclictest275-21dbus-daemon10:50:010
36045199701,65cyclictest457659-21ssh12:20:582
36044499704,61cyclictest150-21pr/ttyS212:37:020
36045199692,36cyclictest448889-21ssh12:02:392
36045199691,64cyclictest368783-21sshd09:14:262
36045199682,61cyclictest450618-21ssh12:06:152
36045199682,36cyclictest371283-21ssh09:19:442
36045199682,36cyclictest371283-21ssh09:19:442
36045199681,63cyclictest463799-21ssh12:34:172
36045199681,61cyclictest445131-21munin-run11:55:022
360444996827,19cyclictest440439-21ssh11:45:010
360444996819,41cyclictest12-21ksoftirqd/010:03:160
36045199671,61cyclictest373114-21ssh09:23:292
36045199662,59cyclictest465495-21ssh12:37:492
36045199661,61cyclictest435261-21ssh11:34:022
36045199661,61cyclictest435261-21ssh11:34:022
36045199661,61cyclictest425496-21ssh11:13:492
36045199661,61cyclictest400240-21ssh10:20:072
36045199661,61cyclictest397550-21sshd10:14:492
36045199661,61cyclictest389039-21ssh09:56:532
36045199661,61cyclictest389039-21ssh09:56:532
36045199661,61cyclictest376430-21ssh09:30:062
36045199661,60cyclictest418778-21sshd10:59:492
36045199661,60cyclictest364013-21sshd08:15:492
36045199651,60cyclictest404465-21ssh10:29:272
36045199651,60cyclictest404465-21ssh10:29:272
36045199651,60cyclictest380118-21ssh09:38:062
36045199651,59cyclictest439154-21ssh11:42:012
36045199641,60cyclictest387371-21ssh09:53:252
36045199641,59cyclictest436941-21ssh11:37:242
36045199641,59cyclictest431996-21ssh11:27:112
36045199641,59cyclictest414968-21ssh10:51:212
36045199641,59cyclictest413594-21ssh10:48:462
36045199641,59cyclictest393205-21ssh10:05:192
36045199641,59cyclictest393205-21ssh10:05:192
36045199641,59cyclictest385333-21ssh09:49:012
36045199641,59cyclictest381433-21ssh09:40:362
360448996428,10cyclictest23-21ksoftirqd/109:38:021
36045199632,57cyclictest407930-21ssh10:36:322
36045199632,56cyclictest362270-21munin-run07:45:012
36045199631,59cyclictest452574-21ssh12:10:142
36045199631,58cyclictest459640-21ssh12:25:092
36045199631,58cyclictest445722-21ssh11:55:522
36045199631,58cyclictest441018-21ssh11:45:582
36045199631,58cyclictest419534-21ssh11:00:512
36045199631,58cyclictest405030-21ssh10:30:202
36045199631,58cyclictest397821-21ssh10:15:022
36045199631,58cyclictest397821-21ssh10:15:022
36045199631,58cyclictest374466-21ssh09:26:022
36045199622,55cyclictest363771-21sshd08:12:492
36045199622,55cyclictest363771-21sshd08:12:482
36045199621,57cyclictest365346-21sshd08:42:502
36045199621,57cyclictest360923-21sshd07:19:492
36045199621,56cyclictest428758-21ssh11:20:072
36045199621,56cyclictest428758-21ssh11:20:072
360444996257,3cyclictest150-21pr/ttyS210:30:010
360444996257,3cyclictest150-21pr/ttyS210:30:010
36045599612,45cyclictest3467602-21systemd-journal10:10:033
36045599612,45cyclictest3467602-21systemd-journal10:10:023
36045199612,55cyclictest361220-21munin-run07:25:022
36045199612,54cyclictest361725-21munin-run07:35:002
360455996017,8cyclictest37-21ksoftirqd/310:03:043
360444996041,4cyclictest16850irq/134-800000011:20:010
360444996041,4cyclictest16850irq/134-800000011:20:010
360444996025,29cyclictest12-21ksoftirqd/010:25:020
360444996017,12cyclictest275-21dbus-daemon11:10:020
360444996017,12cyclictest275-21dbus-daemon11:10:010
360444996013,14cyclictest1-21systemd10:10:010
360444996013,14cyclictest1-21systemd10:10:010
36045199591,54cyclictest364817-21sshd08:33:492
36045199591,54cyclictest361456-21sshd07:29:492
36045199591,53cyclictest365135-21munin-run08:40:012
360444995939,17cyclictest12-21ksoftirqd/010:11:350
36044499592,40cyclictest414424-21ssh10:50:020
360455995820,14cyclictest413281-21ssh10:47:563
36045199581,53cyclictest366393-21sshd09:04:252
36045199581,53cyclictest363307-21munin-run08:05:002
36045199581,53cyclictest360643-21sshd07:10:492
360448995850,6cyclictest23-21ksoftirqd/108:50:011
360444995839,14cyclictest1-21systemd10:20:000
360444995839,14cyclictest1-21systemd10:20:000
360444995837,5cyclictest16850irq/134-800000012:20:000
360444995821,33cyclictest1-21systemd11:15:000
360455995716,8cyclictest389064-21ssh09:56:573
360455995716,8cyclictest389064-21ssh09:56:563
360444995729,12cyclictest385297-21ssh09:48:570
360444995727,14cyclictest442143-21ssh11:48:310
360444995711,31cyclictest12-21ksoftirqd/010:32:560
36045199562,50cyclictest365627-21munin-run08:50:002
36045199562,49cyclictest366640-21sshd09:08:482
36045199562,49cyclictest366640-21sshd09:08:482
36044899565,8cyclictest23-21ksoftirqd/112:04:031
360444995627,13cyclictest451685-21ssh12:08:410
360455995535,17cyclictest366115-21sshd08:59:493
36045199552,48cyclictest363547-21/usr/sbin/munin08:10:002
36045199551,50cyclictest362762-21munin-run07:55:002
36044899557,6cyclictest131rcu_preempt12:16:101
360448995536,15cyclictest275-21dbus-daemon11:25:001
360448995536,15cyclictest275-21dbus-daemon11:25:001
36044899553,11cyclictest23-21ksoftirqd/109:45:581
360444995510,33cyclictest19350irq/34-mmc109:25:010
360455995436,14cyclictest366629-21sshd09:05:493
360455995436,14cyclictest366629-21sshd09:05:493
36045199541,48cyclictest364-21snmpd07:49:072
36045199541,48cyclictest364-21snmpd07:38:332
360448995413,5cyclictest131rcu_preempt10:21:521
360444995427,11cyclictest457409-21ssh12:20:270
360444995427,11cyclictest429043-21ssh11:20:440
360444995427,11cyclictest429043-21ssh11:20:430
360444995426,13cyclictest419756-21ssh11:01:290
360444995425,13cyclictest369094-21ssh09:15:010
36044499542,10cyclictest12-21ksoftirqd/011:31:010
36044499542,10cyclictest12-21ksoftirqd/011:31:000
360455995311,14cyclictest366669-21diskmemload12:39:383
36045199532,47cyclictest364-21snmpd08:57:162
360448995335,13cyclictest3467602-21systemd-journal09:00:001
36044899531,11cyclictest366742-21date09:10:021
36044899531,11cyclictest366742-21date09:10:021
360444995331,15cyclictest447477-21ssh12:00:010
360444995330,19cyclictest462061-21ssh12:30:120
360444995327,11cyclictest444813-21ssh11:54:140
360444995324,13cyclictest417229-21ssh10:56:030
36044499532,36cyclictest369780-21ssh09:16:160
36044499532,36cyclictest369780-21ssh09:16:160
36044499531,11cyclictest120-21kswapd012:05:000
360455995235,14cyclictest427522-21sshd11:17:493
360455995235,14cyclictest427522-21sshd11:17:493
360455995215,13cyclictest383660-21ssh09:45:113
36044899527,10cyclictest23-21ksoftirqd/110:11:481
360444995232,14cyclictest275-21dbus-daemon08:10:010
360444995224,12cyclictest460999-21ssh12:28:200
360444995223,13cyclictest410931-21ssh10:43:000
360444995218,3cyclictest12-21ksoftirqd/009:42:330
360455995134,13cyclictest408669-21ssh10:38:163
360455995134,13cyclictest377289-21ssh09:32:033
360455995133,14cyclictest387026-21ssh09:52:413
360455995130,17cyclictest422177-21ssh11:06:313
360455995130,17cyclictest422177-21ssh11:06:313
360455995115,9cyclictest37-21ksoftirqd/311:22:053
360455995115,9cyclictest37-21ksoftirqd/311:22:043
360451995114,33cyclictest30-21ksoftirqd/208:25:242
36044899514,13cyclictest23-21ksoftirqd/109:54:331
36044899511,9cyclictest0-21swapper/111:25:311
360448995118,6cyclictest131rcu_preempt11:06:591
360448995118,6cyclictest131rcu_preempt11:06:581
360448995114,11cyclictest23-21ksoftirqd/110:02:061
360448995112,9cyclictest23-21ksoftirqd/110:46:301
360444995124,12cyclictest436388-21ssh11:36:050
360444995116,6cyclictest12-21ksoftirqd/009:35:010
36044499511,45cyclictest151rcuc/009:40:010
36045599508,13cyclictest439437-21sshd11:42:483
360455995038,9cyclictest3467602-21systemd-journal10:28:473
360455995038,9cyclictest3467602-21systemd-journal10:28:463
360455995035,12cyclictest402036-21ssh10:24:213
360455995035,12cyclictest380011-21ssh09:37:523
360455995035,11cyclictest448524-21ssh12:01:533
360455995035,11cyclictest416467-21ssh10:54:473
360455995034,14cyclictest433599-21ssh11:30:243
360455995034,14cyclictest433599-21ssh11:30:243
360455995034,13cyclictest440900-21ssh11:45:433
360455995034,13cyclictest420403-21ssh11:03:013
360455995034,12cyclictest457535-21ssh12:20:423
360455995034,12cyclictest410098-21ssh10:41:043
360455995033,13cyclictest452965-21ssh12:11:133
360455995033,13cyclictest445624-21ssh11:55:393
360455995024,22cyclictest433438-21wc11:30:023
360455995013,8cyclictest37-21ksoftirqd/309:40:513
36044899505,6cyclictest131rcu_preempt08:10:001
360448995013,6cyclictest131rcu_preempt10:36:551
360448995012,9cyclictest23-21ksoftirqd/111:03:431
360444995040,5cyclictest16950irq/224-800000007:30:000
360444995039,6cyclictest365099-21munin-plugin-st08:40:000
360444995034,12cyclictest3467602-21systemd-journal09:59:350
360444995034,12cyclictest3467602-21systemd-journal09:59:340
360444995022,13cyclictest432585-21ssh11:28:260
360455994936,10cyclictest460179-21ssh12:26:263
360455994935,11cyclictest462796-21ssh12:31:543
360455994935,11cyclictest456521-21ssh12:18:583
360455994934,12cyclictest451625-21ssh12:08:333
360455994934,12cyclictest370169-21ssh09:17:053
360455994934,12cyclictest370169-21ssh09:17:053
360455994934,11cyclictest362018-21munin-run07:40:013
360455994933,13cyclictest443215-21ssh11:50:363
360455994933,13cyclictest368129-21ssh09:12:563
360455994932,14cyclictest374505-21ssh09:26:163
360455994932,13cyclictest366384-21sshd09:02:503
360455994920,24cyclictest360826-21sed07:15:013
360455994913,5cyclictest37-21ksoftirqd/311:12:223
36045199491,44cyclictest364-21snmpd08:50:032
36044899499,9cyclictest23-21ksoftirqd/110:59:151
36044899494,6cyclictest131rcu_preempt10:33:421
360448994939,7cyclictest23-21ksoftirqd/112:10:071
36044899493,12cyclictest23-21ksoftirqd/111:44:361
36044899493,10cyclictest23-21ksoftirqd/112:29:001
36044899493,10cyclictest23-21ksoftirqd/112:05:151
360448994913,5cyclictest131rcu_preempt12:22:421
36044499491,43cyclictest374087-21ssh09:25:130
360455994835,10cyclictest406949-21ssh10:34:503
360455994835,10cyclictest372091-21ssh09:21:033
360455994834,11cyclictest417304-21ssh10:56:133
360455994833,12cyclictest397181-21ssh10:14:023
360455994832,13cyclictest435958-21ssh11:35:103
360448994838,7cyclictest23-21ksoftirqd/111:45:461
36044899482,9cyclictest23-21ksoftirqd/110:17:171
36044899482,9cyclictest23-21ksoftirqd/110:17:171
36044899481,9cyclictest23-21ksoftirqd/109:42:261
360448994818,23cyclictest131rcu_preempt11:39:071
36044899481,7cyclictest23-21ksoftirqd/109:56:151
36044899481,7cyclictest23-21ksoftirqd/109:56:141
36044899481,7cyclictest0-21swapper/109:31:441
360448994817,6cyclictest131rcu_preempt08:42:491
360448994813,11cyclictest23-21ksoftirqd/110:06:461
360448994813,11cyclictest23-21ksoftirqd/110:06:461
360444994829,14cyclictest275-21dbus-daemon08:25:000
360444994810,22cyclictest19350irq/34-mmc108:45:020
360455994735,9cyclictest363256-21sshd08:00:493
360455994733,10cyclictest365640-21munin-run08:50:023
360455994732,12cyclictest397837-21ssh10:15:073
360455994732,12cyclictest397837-21ssh10:15:063
360455994732,12cyclictest362995-21sshd07:59:163
360455994726,16cyclictest361187-21sendmail-mta07:20:593
36044899478,6cyclictest131rcu_preempt10:51:461
36044899477,6cyclictest131rcu_preempt09:24:391
36044899476,8cyclictest23-21ksoftirqd/111:50:251
360448994737,6cyclictest363016-21modprobe08:00:001
360448994717,7cyclictest131rcu_preempt11:17:071
360448994717,7cyclictest131rcu_preempt11:17:071
360448994711,7cyclictest131rcu_preempt11:10:151
360455994634,9cyclictest365852-21sshd08:52:493
360455994633,10cyclictest361694-21sshd07:31:493
360455994628,13cyclictest1-21systemd07:55:003
36044899468,6cyclictest131rcu_preempt12:34:491
360448994637,6cyclictest23-21ksoftirqd/107:50:011
360448994618,9cyclictest23-21ksoftirqd/110:27:401
360448994618,9cyclictest23-21ksoftirqd/110:27:391
360448994615,6cyclictest131rcu_preempt12:35:031
360448994614,9cyclictest23-21ksoftirqd/109:11:191
360455994532,10cyclictest365057-21sshd08:35:483
360455994532,10cyclictest362511-21munin-run07:50:013
360455994519,21cyclictest364220-21date08:20:023
36045199452,37cyclictest360526-21wc07:10:022
36045199452,37cyclictest360526-21wc07:10:022
360448994529,12cyclictest275-21dbus-daemon07:40:011
360448994517,6cyclictest131rcu_preempt10:44:301
360448994515,5cyclictest131rcu_preempt11:30:321
360448994515,5cyclictest131rcu_preempt11:30:311
360444994520,21cyclictest12-21ksoftirqd/007:15:000
36044499451,12cyclictest12-21ksoftirqd/012:13:560
360455994433,8cyclictest362236-21sshd07:44:153
360455994432,9cyclictest364825-21sshd08:34:493
36044899447,11cyclictest23-21ksoftirqd/109:25:501
36044899443,5cyclictest131rcu_preempt11:58:411
360448994434,7cyclictest23-21ksoftirqd/107:30:001
360448994413,6cyclictest131rcu_preempt09:18:191
360448994413,6cyclictest131rcu_preempt09:18:191
360444994437,5cyclictest19350irq/34-mmc109:05:020
360444994427,12cyclictest275-21dbus-daemon08:35:000
360455994332,8cyclictest364554-21sshd08:25:493
360455994322,16cyclictest3467602-21systemd-journal08:15:003
360455994322,16cyclictest3467602-21systemd-journal08:15:003
36044899431,5cyclictest0-21swapper/108:34:481
360444994328,12cyclictest12-21ksoftirqd/009:10:020
360444994328,12cyclictest12-21ksoftirqd/009:10:010
360444994320,17cyclictest361974-21modprobe07:40:000
360455994229,10cyclictest364-21snmpd08:43:563
360455994225,14cyclictest364-21snmpd07:17:333
36044899429,5cyclictest131rcu_preempt07:10:021
360444994227,11cyclictest3467602-21systemd-journal07:53:500
360455994134,4cyclictest363560-21munin-run08:10:013
360455994126,12cyclictest3467562-21sshd07:26:503
36044899417,5cyclictest131rcu_preempt08:25:021
360448994129,8cyclictest23-21ksoftirqd/108:15:011
360448994129,8cyclictest23-21ksoftirqd/108:15:011
360448994129,7cyclictest23-21ksoftirqd/108:20:001
36044899410,7cyclictest150-21pr/ttyS207:21:331
36044899410,5cyclictest150-21pr/ttyS207:19:131
360444994131,7cyclictest364020-21cron08:16:590
360444994129,8cyclictest360446-21ls07:10:010
360444994129,8cyclictest360446-21ls07:10:010
360444994122,14cyclictest361690-21sshd07:30:490
360448994028,7cyclictest309-21in:imuxsock07:15:001
360448994025,11cyclictest3467602-21systemd-journal07:54:151
36044499402,12cyclictest1-21systemd07:48:490
360444994020,14cyclictest275-21dbus-daemon08:55:010
36044499401,34cyclictest362227-21sshd07:41:490
360455993921,14cyclictest364325-21cron08:25:003
36044899397,5cyclictest131rcu_preempt08:05:001
36044899393,11cyclictest23-21ksoftirqd/109:04:001
360444993923,12cyclictest12-21ksoftirqd/007:20:000
36044499391,6cyclictest3467602-21systemd-journal09:00:000
360455993818,14cyclictest360525-21grep07:10:023
360455993818,14cyclictest360525-21grep07:10:023
36044899381,7cyclictest150-21pr/ttyS208:51:021
36044899381,6cyclictest150-21pr/ttyS208:25:031
360444993820,8cyclictest12-21ksoftirqd/008:00:020
360444993725,10cyclictest12-21ksoftirqd/008:29:490
36044499371,13cyclictest0-21swapper/008:05:000
360448993623,9cyclictest23-21ksoftirqd/108:40:011
36044899360,6cyclictest150-21pr/ttyS207:32:151
360444993622,9cyclictest309-21in:imuxsock08:40:020
360448993511,10cyclictest23-21ksoftirqd/107:42:501
360444993528,4cyclictest16950irq/224-800000008:13:500
360444993528,4cyclictest16950irq/224-800000008:13:500
360448993120,9cyclictest23-21ksoftirqd/107:10:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional