You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-14 - 20:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Fri Aug 14, 2026 12:45:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
728154999732,18cyclictest786007-21sh10:59:231
728151999178,3cyclictest0-21swapper/007:35:020
728154998829,22cyclictest736397-21sh09:14:141
728154998829,22cyclictest736397-21sh09:14:141
728154998528,19cyclictest824105-21ssh12:20:031
728154998528,17cyclictest754620-21ssh09:53:041
728154998524,26cyclictest779843-21ssh10:46:001
728154998427,19cyclictest761048-21ssh10:06:171
728154998427,19cyclictest761048-21ssh10:06:171
728154998426,22cyclictest826493-21ssh12:25:171
728154998327,17cyclictest764094-21ssh10:13:041
728154998123,20cyclictest792130-21sh11:12:191
728154998029,17cyclictest813498-21sh11:57:471
728154997932,14cyclictest756162-21sh09:56:071
728154997927,16cyclictest730319-21wc07:50:021
728154997925,17cyclictest820580-21ssh12:12:551
728154997829,21cyclictest815361-21sh12:01:401
728154997736,13cyclictest779001-21ssh10:44:311
728154997736,13cyclictest779001-21ssh10:44:311
728154997734,16cyclictest364-21snmpd11:17:251
728154997729,17cyclictest744489-21sh09:31:001
728154997728,15cyclictest787294-21ssh11:01:521
728154997727,16cyclictest748696-21sed09:40:031
728154997727,16cyclictest748696-21sed09:40:021
728154997725,15cyclictest775723-21ssh10:37:241
728154997724,16cyclictest759921-21ssh10:04:181
728154997724,13cyclictest828936-21ssh12:30:231
728154997626,15cyclictest802456-21ssh11:34:181
728154997625,17cyclictest738177-21ssh09:17:541
728154997525,16cyclictest809033-21sh11:48:111
728154997525,15cyclictest734086-21munin-run09:05:001
728154997525,15cyclictest734086-21munin-run09:05:001
728154997523,15cyclictest805472-21ssh11:40:221
728154997432,14cyclictest768463-21ssh10:22:131
728154997432,14cyclictest768463-21ssh10:22:131
728154997424,14cyclictest833142-21ssh12:39:481
728154997422,21cyclictest765382-21rm10:15:291
728154997332,14cyclictest150-21pr/ttyS208:11:281
728154997322,15cyclictest771674-21ssh10:29:061
728161997240,5cyclictest721233-21kworker/2:110:36:592
728154997229,14cyclictest0-21swapper/111:37:311
728154997224,14cyclictest774291-21ssh10:34:431
728154997223,16cyclictest743257-21ssh09:28:351
728154997221,18cyclictest811821-21ssh11:54:191
728154997221,14cyclictest790000-21ssh11:07:491
728154997130,17cyclictest749021-21sh09:40:311
728154997026,14cyclictest799780-21sh11:28:331
728154997026,13cyclictest751302-21rm09:45:451
728154997026,13cyclictest751302-21rm09:45:441
728154997023,14cyclictest728849-21sed07:20:021
72815199707,39cyclictest732230-21kworker/u8:109:15:020
72815199707,39cyclictest732230-21kworker/u8:109:15:020
728154996923,13cyclictest797182-21ssh11:23:001
728154996922,13cyclictest781942-21ssh10:50:301
728161996841,7cyclictest721233-21kworker/2:112:36:352
728154996730,11cyclictest0-21swapper/112:17:411
728154996730,11cyclictest0-21swapper/112:17:401
728154996728,12cyclictest150-21pr/ttyS208:43:191
728154996527,12cyclictest150-21pr/ttyS209:06:551
72815199652,51cyclictest0-21swapper/010:00:020
728165996438,15cyclictest37-21ksoftirqd/311:48:323
728154996426,12cyclictest739667-21ssh09:20:531
728154996425,12cyclictest150-21pr/ttyS207:21:351
728151996442,19cyclictest792253-21sshd11:12:330
728154996321,12cyclictest817897-21ssh12:07:061
728165996210,14cyclictest37-21ksoftirqd/309:30:013
728165996016,10cyclictest37-21ksoftirqd/309:55:003
728154996043,14cyclictest729372-21sshd07:32:331
72815199602,41cyclictest780294-21ssh10:47:100
728154995945,10cyclictest728352-21munin-run07:15:011
728154995945,10cyclictest728352-21munin-run07:15:011
728151995947,8cyclictest12-21ksoftirqd/012:34:420
728165995815,12cyclictest37-21ksoftirqd/310:10:003
728165995815,12cyclictest37-21ksoftirqd/310:10:003
728154995844,11cyclictest733785-21sshd08:59:311
728154995838,15cyclictest364-21snmpd08:03:001
728154995838,15cyclictest364-21snmpd08:03:001
728151995822,33cyclictest12-21ksoftirqd/010:41:020
728151995822,33cyclictest12-21ksoftirqd/010:41:020
728151995817,10cyclictest1-21systemd09:45:000
728154995745,9cyclictest364-21snmpd08:48:551
728154995745,9cyclictest364-21snmpd08:48:551
728151995738,16cyclictest16850irq/134-800000011:29:120
728151995730,23cyclictest796294-21ssh11:20:540
728151995727,14cyclictest765938-21ssh10:16:490
728154995644,8cyclictest310-21in:imklog08:31:411
728154995644,8cyclictest310-21in:imklog08:31:411
728154995643,9cyclictest730449-21munin-run07:55:011
728154995643,9cyclictest730449-21munin-run07:55:011
728154995643,10cyclictest730652-21sshd07:55:331
728154995642,9cyclictest48-21kcompactd008:18:331
728154995642,9cyclictest48-21kcompactd008:18:331
728154995639,13cyclictest3467602-21systemd-journal07:28:391
72815199564,48cyclictest0-21swapper/009:37:050
72815199564,48cyclictest0-21swapper/009:37:050
728151995634,17cyclictest275-21dbus-daemon12:40:010
728151995628,14cyclictest771619-21ssh10:29:000
728151995628,13cyclictest808558-21ssh11:47:010
728151995619,33cyclictest741734-21cat09:25:030
728161995546,6cyclictest721233-21kworker/2:110:41:442
728161995546,6cyclictest721233-21kworker/2:110:41:442
728161995546,5cyclictest743882-21kworker/2:311:07:072
728161995520,12cyclictest30-21ksoftirqd/210:20:012
728151995527,24cyclictest7605641ssh10:05:170
728151995527,24cyclictest7605641ssh10:05:170
728151995527,12cyclictest786993-21ssh11:01:100
728151995525,16cyclictest743873-21ssh09:30:000
728151995524,28cyclictest12-21ksoftirqd/009:18:100
728161995445,5cyclictest721233-21kworker/2:107:20:002
728154995441,10cyclictest364-21snmpd08:35:041
728151995429,10cyclictest774144-21ssh10:34:180
728151995426,13cyclictest804652-21ssh11:38:590
728151995426,13cyclictest745729-21ssh09:33:550
728151995425,14cyclictest776290-21ssh10:38:460
728151995420,28cyclictest817307-21sh12:05:410
728151995419,30cyclictest734339-21diskmemload11:55:030
72816199539,5cyclictest131rcu_preempt09:15:022
72816199539,5cyclictest131rcu_preempt09:15:012
728161995345,5cyclictest743882-21kworker/2:312:10:092
728161995345,5cyclictest743882-21kworker/2:312:00:572
728161995345,5cyclictest733063-21kworker/2:009:24:282
728161995343,6cyclictest721233-21kworker/2:111:35:392
728161995343,6cyclictest721233-21kworker/2:110:10:032
728161995343,6cyclictest721233-21kworker/2:110:10:022
728161995343,6cyclictest721233-21kworker/2:109:26:392
728161995321,8cyclictest131rcu_preempt11:45:482
728161995314,11cyclictest30-21ksoftirqd/212:10:002
728154995341,8cyclictest3467602-21systemd-journal08:52:011
728154995340,9cyclictest364-21snmpd07:44:421
728151995326,11cyclictest822359-21ssh12:16:370
728151995326,11cyclictest822359-21ssh12:16:360
728151995325,13cyclictest794745-21ssh11:17:500
72816599529,5cyclictest131rcu_preempt08:20:003
72816599529,5cyclictest131rcu_preempt08:20:003
72816599528,5cyclictest131rcu_preempt12:10:013
72816599525,6cyclictest131rcu_preempt11:40:023
728161995244,5cyclictest743882-21kworker/2:311:32:392
728161995244,5cyclictest743882-21kworker/2:311:12:472
728161995244,5cyclictest721233-21kworker/2:109:55:192
728161995243,6cyclictest721233-21kworker/2:110:30:112
728161995243,6cyclictest721233-21kworker/2:109:51:182
728154995231,16cyclictest728238-21latency_hist07:10:021
728151995226,11cyclictest819411-21ssh12:10:080
728151995224,14cyclictest764140-21ssh10:13:090
72815199522,3cyclictest131rcu_preempt09:49:390
72815199522,3cyclictest131rcu_preempt09:49:380
728165995113,5cyclictest131rcu_preempt11:10:003
728161995143,5cyclictest743882-21kworker/2:311:23:002
728161995143,5cyclictest743882-21kworker/2:311:16:302
728161995143,5cyclictest721233-21kworker/2:111:00:282
728161995143,5cyclictest721233-21kworker/2:110:10:142
728161995143,5cyclictest721233-21kworker/2:110:04:212
728161995142,6cyclictest721233-21kworker/2:112:31:012
728161995142,6cyclictest721233-21kworker/2:110:55:172
728161995142,6cyclictest721233-21kworker/2:109:15:512
728154995140,8cyclictest48-21kcompactd007:35:171
728154995140,8cyclictest150-21pr/ttyS208:26:341
728154995137,10cyclictest364-21snmpd08:05:181
728154995137,10cyclictest364-21snmpd08:05:181
72815199514,3cyclictest131rcu_preempt12:24:520
728151995140,5cyclictest16950irq/224-800000011:00:000
728151995135,11cyclictest3467602-21systemd-journal07:10:020
728151995135,11cyclictest3467602-21systemd-journal07:10:020
728151995123,14cyclictest827151-21ssh12:26:470
728151995119,29cyclictest12-21ksoftirqd/011:57:540
72816199509,6cyclictest131rcu_preempt09:00:002
728161995042,5cyclictest743882-21kworker/2:312:25:552
728161995042,5cyclictest743882-21kworker/2:311:29:092
728161995042,5cyclictest721233-21kworker/2:111:40:092
728161995042,5cyclictest721233-21kworker/2:110:52:252
728161995042,5cyclictest721233-21kworker/2:110:26:152
728161995042,5cyclictest721233-21kworker/2:109:48:212
728161995042,5cyclictest721233-21kworker/2:109:48:212
728161995042,5cyclictest721233-21kworker/2:109:40:202
728161995042,5cyclictest721233-21kworker/2:109:36:342
728161995042,5cyclictest721233-21kworker/2:109:36:342
728161995042,4cyclictest743882-21kworker/2:310:45:142
728161995040,6cyclictest743882-21kworker/2:311:57:062
728161995034,12cyclictest275-21dbus-daemon12:20:012
728161995034,12cyclictest275-21dbus-daemon12:20:012
728151995030,3cyclictest131rcu_preempt12:02:200
728151995020,15cyclictest755110-21ssh09:54:170
728165994910,5cyclictest131rcu_preempt10:18:213
728161994941,5cyclictest743882-21kworker/2:310:22:162
728161994941,5cyclictest743882-21kworker/2:310:22:162
728161994941,5cyclictest721233-21kworker/2:112:21:292
728161994941,5cyclictest721233-21kworker/2:109:33:002
728154994936,9cyclictest364-21snmpd08:24:071
72815199493,3cyclictest131rcu_preempt11:08:410
72815199491,33cyclictest801595-21ssh11:32:200
72816599485,13cyclictest37-21ksoftirqd/308:10:013
72816599485,13cyclictest37-21ksoftirqd/308:10:013
72816599484,6cyclictest131rcu_preempt11:12:193
72816599484,5cyclictest131rcu_preempt10:36:213
728165994837,8cyclictest774227-21sshd10:34:333
728165994821,11cyclictest37-21ksoftirqd/308:55:003
728165994812,7cyclictest131rcu_preempt12:18:143
728165994812,7cyclictest131rcu_preempt12:18:143
728161994840,5cyclictest721233-21kworker/2:107:44:022
728161994840,5cyclictest721233-21kworker/2:107:39:572
72816599477,6cyclictest131rcu_preempt12:00:013
72816599476,6cyclictest131rcu_preempt10:12:203
72816599475,5cyclictest131rcu_preempt09:50:013
72816599475,5cyclictest131rcu_preempt09:50:013
72816599471,6cyclictest131rcu_preempt10:05:003
72816599471,5cyclictest0-21swapper/309:32:473
728165994710,6cyclictest131rcu_preempt08:50:003
728165994710,6cyclictest131rcu_preempt08:50:003
728165994710,5cyclictest131rcu_preempt11:42:103
72816199476,5cyclictest131rcu_preempt08:50:012
72816199476,5cyclictest131rcu_preempt08:50:002
728161994739,5cyclictest721233-21kworker/2:111:50:332
728161994739,5cyclictest721233-21kworker/2:108:25:342
728161994739,5cyclictest721233-21kworker/2:108:04:202
728161994739,5cyclictest721233-21kworker/2:108:04:202
728161994739,5cyclictest721233-21kworker/2:107:51:332
728161994739,5cyclictest721233-21kworker/2:107:51:332
728161994739,5cyclictest721233-21kworker/2:107:26:592
728161994739,5cyclictest721233-21kworker/2:107:10:182
728161994739,5cyclictest721233-21kworker/2:107:10:182
728161994738,5cyclictest721233-21kworker/2:108:42:242
72815199478,30cyclictest12-21ksoftirqd/008:05:020
72815199478,30cyclictest12-21ksoftirqd/008:05:010
72815199477,36cyclictest48-21kcompactd010:55:020
728151994720,11cyclictest768352-21ssh10:21:460
728151994720,11cyclictest768352-21ssh10:21:460
72816599468,6cyclictest131rcu_preempt07:38:333
72816599466,5cyclictest131rcu_preempt09:59:553
72816599463,5cyclictest131rcu_preempt11:23:543
72816599463,4cyclictest131rcu_preempt11:33:043
72816599462,5cyclictest131rcu_preempt10:30:003
72816599462,4cyclictest131rcu_preempt12:14:503
72816599461,5cyclictest131rcu_preempt09:43:013
728165994611,5cyclictest131rcu_preempt12:01:223
728165994610,5cyclictest131rcu_preempt11:15:263
728161994638,5cyclictest721233-21kworker/2:108:50:162
728161994637,6cyclictest721233-21kworker/2:107:25:022
72815199465,3cyclictest131rcu_preempt07:45:010
72815199464,4cyclictest131rcu_preempt10:03:210
728151994638,4cyclictest16950irq/224-800000008:00:000
72816599458,5cyclictest131rcu_preempt09:05:013
72816599458,5cyclictest131rcu_preempt09:05:013
72816599457,7cyclictest131rcu_preempt09:00:003
72816599457,6cyclictest131rcu_preempt10:55:013
72816599454,5cyclictest131rcu_preempt10:55:063
72816599453,5cyclictest131rcu_preempt11:30:013
72816599453,5cyclictest131rcu_preempt10:48:583
728165994532,9cyclictest3467602-21systemd-journal07:17:003
72816599452,39cyclictest1-21systemd07:45:003
72816599451,9cyclictest0-21swapper/311:03:423
72816599451,7cyclictest0-21swapper/310:42:403
72816599451,7cyclictest0-21swapper/310:42:393
728165994513,5cyclictest131rcu_preempt12:21:173
728165994512,8cyclictest131rcu_preempt12:30:013
728165994510,6cyclictest131rcu_preempt09:12:333
728165994510,6cyclictest131rcu_preempt09:12:333
728161994538,4cyclictest731503-21kworker/2:208:30:252
728161994538,4cyclictest731503-21kworker/2:208:30:252
728161994538,4cyclictest721233-21kworker/2:107:30:222
728161994537,5cyclictest721233-21kworker/2:109:07:382
728161994537,5cyclictest721233-21kworker/2:108:15:092
728161994537,5cyclictest721233-21kworker/2:108:15:092
728161994537,5cyclictest721233-21kworker/2:108:11:332
728161994537,5cyclictest721233-21kworker/2:108:08:012
728161994537,5cyclictest721233-21kworker/2:108:08:012
728161994537,5cyclictest721233-21kworker/2:107:55:182
728161994535,6cyclictest721233-21kworker/2:107:45:412
72816199451,5cyclictest131rcu_preempt08:40:012
728151994538,4cyclictest16850irq/134-800000011:45:010
72816599449,5cyclictest131rcu_preempt11:50:123
72816599447,6cyclictest131rcu_preempt07:55:023
72816599447,6cyclictest131rcu_preempt07:55:023
72816599446,6cyclictest131rcu_preempt12:40:023
72816599445,5cyclictest131rcu_preempt08:00:003
72816599444,5cyclictest131rcu_preempt09:36:063
72816599444,5cyclictest131rcu_preempt09:36:063
72816599444,4cyclictest131rcu_preempt09:15:253
728165994431,9cyclictest3467602-21systemd-journal12:35:013
72816599441,7cyclictest0-21swapper/309:21:393
728165994411,5cyclictest131rcu_preempt08:35:013
728165994411,5cyclictest131rcu_preempt08:35:013
72816599435,3cyclictest0-21swapper/308:05:003
72816599435,3cyclictest0-21swapper/308:05:003
728165994333,6cyclictest274-21cron09:10:003
72816599429,4cyclictest131rcu_preempt07:35:013
728165994225,11cyclictest37-21ksoftirqd/310:25:003
728165994225,11cyclictest37-21ksoftirqd/310:25:003
728161994235,4cyclictest721233-21kworker/2:108:21:062
728161994218,9cyclictest30-21ksoftirqd/209:05:022
728161994218,9cyclictest30-21ksoftirqd/209:05:022
72815199425,24cyclictest19350irq/34-mmc107:25:020
728151994236,3cyclictest0-21swapper/007:20:020
72815199422,35cyclictest730419-21sshd07:54:330
72815199422,35cyclictest730419-21sshd07:54:330
72816599412,4cyclictest131rcu_preempt07:50:023
728165994116,5cyclictest131rcu_preempt08:30:023
728151994134,4cyclictest16950irq/224-800000008:08:330
728151994134,4cyclictest16950irq/224-800000008:08:330
728151994120,15cyclictest275-21dbus-daemon09:00:000
72815199411,7cyclictest731735-21cron08:20:000
72815199411,7cyclictest731735-21cron08:20:000
72816599409,4cyclictest131rcu_preempt08:45:013
72816599403,6cyclictest131rcu_preempt07:15:023
72816599403,6cyclictest131rcu_preempt07:15:023
72816599402,9cyclictest37-21ksoftirqd/307:30:033
72815199405,30cyclictest364-21snmpd08:45:590
72815199405,30cyclictest364-21snmpd08:45:590
728151994033,4cyclictest16950irq/224-800000008:35:000
728151994033,4cyclictest16950irq/224-800000008:35:000
728151994029,9cyclictest12-21ksoftirqd/007:25:010
728151994026,10cyclictest733026-21sshd08:43:330
72815199402,35cyclictest734344-21munin-run09:10:000
728151994018,16cyclictest275-21dbus-daemon08:40:000
72816599392,6cyclictest131rcu_preempt08:40:003
728151993921,12cyclictest275-21dbus-daemon08:55:000
72816599386,6cyclictest131rcu_preempt07:25:003
72816199384,4cyclictest131rcu_preempt07:10:022
72816599372,5cyclictest131rcu_preempt08:25:003
72815199373,3cyclictest131rcu_preempt07:10:020
72815199372,4cyclictest1-21systemd07:35:000
72815199372,31cyclictest731978-21sshd08:20:330
72815199368,6cyclictest131rcu_preempt08:15:020
72816599357,5cyclictest131rcu_preempt08:11:333
72816599348,6cyclictest131rcu_preempt07:10:023
72815199341,2cyclictest0-21swapper/008:30:000
72815199341,29cyclictest0-21swapper/009:00:460
72815199341,29cyclictest0-21swapper/009:00:460
72815199321,3cyclictest730159-21sshd07:48:330
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional