You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-30 - 01:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat Aug 29, 2026 12:45:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3478390998513,47cyclictest3523329-21ssh10:33:130
347839099811,63cyclictest3570145-21ssh12:13:190
3478390998114,4cyclictest0-21swapper/010:04:310
3478390997927,48cyclictest3485807-21ssh09:12:460
3478390997920,55cyclictest3539476-21ssh11:07:360
3478390997916,14cyclictest3573516-21ssh12:20:110
347839099791,60cyclictest0-21swapper/011:17:190
3478390997728,44cyclictest3492271-21ssh09:26:320
347839099772,69cyclictest3480625-21sshd07:50:070
3478390997626,32cyclictest3565075-21ssh12:02:090
3478390997620,39cyclictest3520523-21ssh10:27:100
347839099761,58cyclictest3578740-21ssh12:31:330
3478390997535,36cyclictest3517535-21ssh10:20:410
3478390997427,31cyclictest3580752-21ssh12:35:380
3478390997425,32cyclictest3498936-21ssh09:40:370
347839099741,58cyclictest3484528-21kworker/u8:109:20:080
347839099741,58cyclictest3484528-21kworker/u8:109:20:070
3478395997333,8cyclictest23-21ksoftirqd/111:25:551
3478390997323,45cyclictest3478850-21systemctl07:20:010
347839099722,64cyclictest3542059-21ssh11:13:050
3478390997225,30cyclictest0-21swapper/011:56:250
3478390997223,30cyclictest3549045-21ssh11:28:000
3478390997222,32cyclictest3557801-21ssh11:46:340
3478390997124,43cyclictest16950irq/224-800000009:50:020
3478390997037,30cyclictest16950irq/224-800000009:51:230
347839099701,65cyclictest3553957-21ssh11:38:230
3478395996931,29cyclictest23-21ksoftirqd/109:05:011
3478390996927,25cyclictest3533994-21ssh10:55:390
3478390996927,25cyclictest3496996-21ssh09:36:340
347839099692,61cyclictest3512017-21ssh10:09:190
347839099682,61cyclictest3467602-21systemd-journal12:25:150
347839099682,61cyclictest3467602-21systemd-journal12:25:140
3478390996818,33cyclictest3488307-21ssh09:18:060
347839099681,62cyclictest0-21swapper/009:56:210
347839099681,3cyclictest0-21swapper/011:45:030
3478390996811,13cyclictest3516526-21ssh10:18:530
3478390996732,30cyclictest16850irq/134-800000011:25:000
347839099673,16cyclictest3496121-21ssh09:35:010
3478390996721,29cyclictest3530995-21ssh10:49:360
3478400996626,12cyclictest37-21ksoftirqd/310:00:003
3478390996635,15cyclictest3513895-21ssh10:13:180
3478390996635,15cyclictest3513895-21ssh10:13:180
3478390996628,23cyclictest3532828-21ssh10:53:310
347839099662,58cyclictest3480920-21sendmail08:00:010
3478390996623,38cyclictest3524986-21sh10:36:360
3478390996517,2cyclictest0-21swapper/012:07:130
3478390996510,11cyclictest3551920-21ssh11:34:080
3478390996423,35cyclictest1-21systemd07:25:010
3478390996423,35cyclictest1-21systemd07:25:010
347839099631,40cyclictest19350irq/34-mmc109:00:030
347840099629,30cyclictest3481943-21kworker/u8:208:35:013
347839099626,11cyclictest3560294-21ssh11:51:520
347839099621,56cyclictest3527187-21ssh10:41:080
347839099621,56cyclictest3527187-21ssh10:41:080
347840099617,7cyclictest37-21ksoftirqd/310:45:033
347840099617,7cyclictest37-21ksoftirqd/310:45:023
3478390996122,32cyclictest1-21systemd09:10:000
3478390996117,28cyclictest0-21swapper/012:19:190
3478390996117,28cyclictest0-21swapper/012:19:180
347839099602,12cyclictest3482972-21sshd08:37:070
3478400995917,10cyclictest131rcu_preempt07:45:003
3478400995912,6cyclictest131rcu_preempt10:30:023
3478397995949,6cyclictest3483810-21kworker/2:311:25:482
347839099592,14cyclictest3481448-21munin-run08:10:000
3478390995920,36cyclictest3478592-21systemd-run07:15:020
347839099591,54cyclictest0-21swapper/008:55:000
347839099591,2cyclictest0-21swapper/011:01:470
347839099582,18cyclictest3482759-21unin-run08:35:010
347839099582,13cyclictest3479875-21sh07:40:000
3478400995713,12cyclictest37-21ksoftirqd/311:40:023
3478397995749,5cyclictest3486818-21kworker/2:209:51:122
3478397995746,6cyclictest3486818-21kworker/2:209:29:452
347839099573,17cyclictest311-21rs:main0
3478400995635,16cyclictest275-21dbus-daemon08:50:013
3478400995612,5cyclictest131rcu_preempt10:23:473
3478400995610,6cyclictest131rcu_preempt11:10:013
3478397995647,5cyclictest3483810-21kworker/2:311:15:012
3478397995647,5cyclictest3483810-21kworker/2:311:09:232
3478397995647,5cyclictest3483810-21kworker/2:311:02:292
347839099561,4cyclictest0-21swapper/008:02:450
3478400995510,9cyclictest37-21ksoftirqd/312:10:013
3478390995537,15cyclictest0-21swapper/007:35:010
347840099546,13cyclictest37-21ksoftirqd/310:39:593
347840099542,5cyclictest131rcu_preempt09:53:063
3478400995410,6cyclictest131rcu_preempt12:19:443
3478400995410,6cyclictest131rcu_preempt12:19:433
3478397995446,5cyclictest3486818-21kworker/2:212:34:472
3478397995446,5cyclictest3486818-21kworker/2:210:20:082
3478397995446,5cyclictest3486818-21kworker/2:209:56:102
3478397995446,5cyclictest3486818-21kworker/2:209:25:022
3478397995446,5cyclictest3486818-21kworker/2:209:25:012
3478397995446,5cyclictest3483810-21kworker/2:310:26:482
3478397995445,6cyclictest3486818-21kworker/2:209:45:022
3478397995445,6cyclictest3483810-21kworker/2:312:17:552
3478397995445,6cyclictest3483810-21kworker/2:312:17:542
3478390995418,31cyclictest3467602-21systemd-journal08:25:000
347840099538,6cyclictest131rcu_preempt11:16:543
347840099535,6cyclictest131rcu_preempt12:25:003
3478397995346,4cyclictest3486818-21kworker/2:212:39:152
3478397995345,5cyclictest3483810-21kworker/2:310:47:462
3478397995345,5cyclictest3483810-21kworker/2:309:16:012
3478397995344,6cyclictest3483810-21kworker/2:311:31:412
347839099531,18cyclictest3478394-21sed07:10:010
347840099527,5cyclictest131rcu_preempt10:16:593
3478400995240,8cyclictest37-21ksoftirqd/310:55:003
3478397995244,5cyclictest3486818-21kworker/2:209:30:492
3478397995244,5cyclictest3227916-21kworker/2:007:50:022
3478397995243,6cyclictest3227916-21kworker/2:008:15:002
3478397995243,5cyclictest3486818-21kworker/2:210:04:182
3478397995243,5cyclictest3483810-21kworker/2:311:48:152
3478397995242,6cyclictest3227916-21kworker/2:007:40:022
3478395995236,12cyclictest3467602-21systemd-journal09:20:001
3478395995214,9cyclictest131rcu_preempt10:10:321
3478395995214,9cyclictest131rcu_preempt10:10:321
347839099525,9cyclictest3480147-21modprobe07:45:010
347839099522,13cyclictest3480384-21sshd07:47:070
347839099522,12cyclictest3482476-21sshd08:29:070
347839099522,12cyclictest3482476-21sshd08:29:070
347839099521,12cyclictest3479349-21sshd07:27:080
347840099519,12cyclictest37-21ksoftirqd/309:40:013
347840099517,10cyclictest37-21ksoftirqd/311:50:013
3478397995144,4cyclictest3483810-21kworker/2:312:21:572
3478397995143,5cyclictest3486818-21kworker/2:210:38:262
3478397995143,5cyclictest3483810-21kworker/2:310:59:092
3478397995143,5cyclictest3483810-21kworker/2:310:40:102
3478397995143,5cyclictest3483810-21kworker/2:310:40:092
3478397995143,5cyclictest3227916-21kworker/2:009:11:222
3478397995141,6cyclictest3483810-21kworker/2:311:38:452
3478395995134,12cyclictest275-21dbus-daemon11:35:021
347839099514,13cyclictest3483278-21munin-run08:45:010
347839099511,46cyclictest0-21swapper/008:45:070
347840099503,7cyclictest37-21ksoftirqd/311:05:013
3478400995019,10cyclictest37-21ksoftirqd/312:30:023
3478400995019,10cyclictest37-21ksoftirqd/312:30:023
3478397995042,5cyclictest3486818-21kworker/2:210:08:002
3478397995042,5cyclictest3486818-21kworker/2:209:48:572
3478397995042,5cyclictest3483810-21kworker/2:312:03:192
3478397995042,5cyclictest3483810-21kworker/2:310:30:252
3478397995042,5cyclictest3227916-21kworker/2:007:24:072
3478397995042,5cyclictest3227916-21kworker/2:007:24:072
3478397995041,5cyclictest3486818-21kworker/2:209:37:482
3478397995041,5cyclictest3483810-21kworker/2:311:17:252
3478397995040,6cyclictest3483810-21kworker/2:312:06:332
3478397995040,6cyclictest3483810-21kworker/2:311:55:092
3478390995043,4cyclictest16950irq/224-800000008:20:000
347839099502,7cyclictest3484043-21cut09:00:010
347840099499,5cyclictest131rcu_preempt10:31:313
3478400994935,11cyclictest3578169-21sshd12:30:063
3478400994933,11cyclictest3467602-21systemd-journal11:14:593
3478400994917,10cyclictest0-21swapper/311:25:593
3478397994941,5cyclictest3483810-21kworker/2:312:13:192
3478397994941,5cyclictest3227916-21kworker/2:008:23:082
3478397994941,5cyclictest3227916-21kworker/2:007:15:592
3478397994940,6cyclictest3483810-21kworker/2:312:27:382
3478397994940,6cyclictest3483810-21kworker/2:312:27:382
3478397994940,6cyclictest3483810-21kworker/2:311:40:362
3478397994940,6cyclictest3483810-21kworker/2:310:50:102
3478397994940,5cyclictest3483810-21kworker/2:311:21:112
3478395994932,13cyclictest3467602-21systemd-journal08:50:001
347840099484,23cyclictest131rcu_preempt09:45:023
347840099482,10cyclictest37-21ksoftirqd/311:40:003
3478397994841,5cyclictest3227916-21kworker/2:007:55:012
3478397994840,5cyclictest3227916-21kworker/2:008:40:012
3478397994839,6cyclictest3227916-21kworker/2:007:43:072
3478397994824,17cyclictest275-21dbus-daemon08:00:012
3478395994829,14cyclictest275-21dbus-daemon10:25:471
347840099473,8cyclictest37-21ksoftirqd/309:31:373
3478400994737,7cyclictest37-21ksoftirqd/310:58:363
3478400994715,6cyclictest131rcu_preempt10:01:243
3478400994711,6cyclictest131rcu_preempt12:40:013
3478397994740,4cyclictest3486818-21kworker/2:210:11:172
3478397994740,4cyclictest3486818-21kworker/2:210:11:162
3478397994739,5cyclictest3486818-21kworker/2:210:15:092
3478395994730,13cyclictest3467602-21systemd-journal09:00:011
347840099467,7cyclictest37-21ksoftirqd/310:49:043
3478400994636,7cyclictest37-21ksoftirqd/312:10:433
347840099462,5cyclictest131rcu_preempt11:52:503
3478400994614,5cyclictest131rcu_preempt09:15:013
347840099461,18cyclictest0-21swapper/309:47:063
3478400994611,5cyclictest131rcu_preempt10:09:133
3478397994638,5cyclictest3483810-21kworker/2:311:54:302
3478395994636,5cyclictest23-21ksoftirqd/107:20:011
3478395994626,13cyclictest275-21dbus-daemon09:10:021
347840099459,6cyclictest131rcu_preempt08:30:013
347840099459,6cyclictest131rcu_preempt08:30:003
347840099455,9cyclictest361rcuc/307:10:073
347840099455,5cyclictest131rcu_preempt10:11:093
347840099455,5cyclictest131rcu_preempt10:11:083
3478400994532,9cyclictest37-21ksoftirqd/309:20:013
3478400994531,10cyclictest3480631-21sshd07:50:123
3478400994527,12cyclictest3478864-21tr07:20:003
3478400994515,6cyclictest131rcu_preempt09:21:533
3478400994515,6cyclictest131rcu_preempt09:21:523
3478400994512,8cyclictest37-21ksoftirqd/312:03:493
3478400994511,12cyclictest37-21ksoftirqd/311:31:293
3478397994537,5cyclictest3227916-21kworker/2:008:34:072
3478397994537,5cyclictest3227916-21kworker/2:008:05:022
3478397994535,6cyclictest3227916-21kworker/2:008:45:022
347839599454,27cyclictest3501621-21kworker/u8:009:55:021
3478395994528,11cyclictest275-21dbus-daemon12:35:001
347840099441,8cyclictest0-21swapper/309:30:003
3478400994416,6cyclictest131rcu_preempt11:20:553
3478400994411,6cyclictest131rcu_preempt09:10:003
3478397994435,5cyclictest3227916-21kworker/2:008:28:052
3478397994435,5cyclictest3227916-21kworker/2:008:28:052
347839599446,9cyclictest23-21ksoftirqd/111:43:261
3478395994428,12cyclictest3467602-21systemd-journal09:10:021
347840099439,5cyclictest131rcu_preempt09:00:023
347840099437,10cyclictest37-21ksoftirqd/308:07:063
347840099431,6cyclictest0-21swapper/309:05:013
3478400994312,6cyclictest131rcu_preempt08:40:023
3478397994335,5cyclictest3227916-21kworker/2:008:59:082
3478397994334,5cyclictest3478599-21latency_hist07:15:022
3478397994334,5cyclictest3227916-21kworker/2:007:35:012
3478397994323,15cyclictest3467602-21systemd-journal08:55:002
347839599437,9cyclictest23-21ksoftirqd/112:03:421
3478395994330,10cyclictest23-21ksoftirqd/109:33:061
3478395994328,11cyclictest275-21dbus-daemon12:20:021
3478395994328,11cyclictest275-21dbus-daemon12:20:011
3478395994328,11cyclictest1-21systemd09:45:011
3478395994315,10cyclictest0-21swapper/111:35:061
347840099428,7cyclictest37-21ksoftirqd/311:55:363
347840099425,16cyclictest131rcu_preempt08:42:503
3478400994226,12cyclictest37-21ksoftirqd/307:35:013
347839599423,6cyclictest3561760-21latency_hist11:55:031
3478395994229,10cyclictest3467602-21systemd-journal11:56:021
3478395994229,10cyclictest23-21ksoftirqd/112:15:001
3478400994128,11cyclictest37-21ksoftirqd/308:25:003
3478400994111,7cyclictest131rcu_preempt07:40:003
3478397994129,9cyclictest30-21ksoftirqd/209:10:022
3478397994125,11cyclictest275-21dbus-daemon07:30:002
3478397994124,9cyclictest309-21in:imuxsock08:49:592
3478395994124,11cyclictest275-21dbus-daemon07:55:001
3478400994029,8cyclictest37-21ksoftirqd/307:30:013
3478397994032,5cyclictest3227916-21kworker/2:009:03:482
3478397994027,10cyclictest3467602-21systemd-journal08:15:072
3478395994030,6cyclictest3493783-21ssh09:30:021
3478395994028,9cyclictest275-21dbus-daemon07:50:021
3478395994028,9cyclictest23-21ksoftirqd/109:45:451
3478395994027,10cyclictest23-21ksoftirqd/112:29:021
3478395994027,10cyclictest23-21ksoftirqd/112:29:011
3478395994027,10cyclictest23-21ksoftirqd/111:17:291
3478395994026,9cyclictest0-21swapper/112:40:031
3478395994026,10cyclictest3526467-21cron10:39:591
3478395994023,13cyclictest275-21dbus-daemon08:30:011
3478395994023,13cyclictest275-21dbus-daemon08:30:011
347839599401,3cyclictest131rcu_preempt11:01:331
3478395993932,4cyclictest3468525-21sendmail-mta10:21:021
3478395993930,6cyclictest23-21ksoftirqd/111:15:021
3478395993927,9cyclictest23-21ksoftirqd/111:25:001
3478395993927,9cyclictest23-21ksoftirqd/109:57:321
3478395993926,10cyclictest23-21ksoftirqd/111:10:011
3478395993924,11cyclictest275-21dbus-daemon11:50:001
3478395993924,10cyclictest309-21in:imuxsock07:35:021
3478395993918,16cyclictest1-21systemd08:15:001
347840099388,5cyclictest131rcu_preempt08:05:013
347840099386,6cyclictest131rcu_preempt08:15:013
347840099381,6cyclictest150-21pr/ttyS208:54:323
347840099381,6cyclictest150-21pr/ttyS208:18:343
347840099381,4cyclictest131rcu_preempt08:00:003
3478397993814,6cyclictest291rcuc/208:10:022
3478395993829,6cyclictest23-21ksoftirqd/112:22:411
3478395993826,9cyclictest23-21ksoftirqd/110:54:491
3478395993826,7cyclictest23-21ksoftirqd/107:25:011
3478395993826,7cyclictest23-21ksoftirqd/107:25:001
3478395993824,11cyclictest23-21ksoftirqd/110:45:011
3478395993824,11cyclictest23-21ksoftirqd/110:45:001
3478395993813,6cyclictest131rcu_preempt12:06:361
347840099374,9cyclictest37-21ksoftirqd/307:50:023
3478395993728,6cyclictest23-21ksoftirqd/107:15:021
3478395993728,5cyclictest3491584-21latency_hist09:25:021
3478395993728,5cyclictest3491584-21latency_hist09:25:021
3478395993727,8cyclictest23-21ksoftirqd/110:02:491
3478395993727,6cyclictest3514940-21sh10:15:141
3478395993720,12cyclictest275-21dbus-daemon08:05:021
3478400993618,14cyclictest275-21dbus-daemon07:25:023
3478400993618,14cyclictest275-21dbus-daemon07:25:013
347839599364,4cyclictest131rcu_preempt07:30:011
3478395993629,4cyclictest3529808-21sh10:46:531
3478395993629,4cyclictest3522977-21sh10:32:201
3478395993624,9cyclictest23-21ksoftirqd/108:07:061
3478400993519,12cyclictest3478464-21wc07:10:023
3478395993528,4cyclictest3483259-21sshd08:44:071
3478395993525,8cyclictest23-21ksoftirqd/109:39:421
3478395993524,7cyclictest309-21in:imuxsock11:00:001
3478395993522,9cyclictest23-21ksoftirqd/108:40:011
347839599351,32cyclictest23-21ksoftirqd/110:05:441
3478395993426,5cyclictest3481949-21cron08:20:001
3478395993421,10cyclictest23-21ksoftirqd/108:51:081
3478395993419,11cyclictest3480348-21cat07:45:021
3478395993414,16cyclictest3467602-21systemd-journal08:00:001
3478395993322,9cyclictest23-21ksoftirqd/107:34:431
3478395993320,10cyclictest23-21ksoftirqd/108:32:481
3478395993222,7cyclictest23-21ksoftirqd/108:20:021
3478397993024,4cyclictest3478322-21latency_hist07:10:022
3478395992619,4cyclictest3478322-21latency_hist07:10:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional