You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-13 - 22:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Thu Aug 13, 2026 12:45:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
397870991011,3cyclictest0-21swapper/009:45:020
397870991011,3cyclictest0-21swapper/009:45:020
397870998624,13cyclictest443672-21ssh10:32:430
397880998369,11cyclictest30-21ksoftirqd/210:45:022
397870998228,49cyclictest412886-21ssh09:28:290
397870998228,37cyclictest455351-21ssh10:57:090
397870998228,37cyclictest455351-21ssh10:57:090
39787099821,62cyclictest3467602-21systemd-journal12:09:240
397870998118,6cyclictest16850irq/134-800000012:11:420
39787099801,61cyclictest0-21swapper/010:45:020
397870997824,14cyclictest439243-21ssh10:23:320
397870997824,14cyclictest439243-21ssh10:23:320
397870997820,16cyclictest441269-21ssh10:27:500
397870997726,34cyclictest467475-21ssh11:22:300
39787099752,8cyclictest435249-21latency_hist10:15:020
397870997426,30cyclictest483800-21ssh11:56:350
397870997424,12cyclictest478330-21ssh11:45:060
39787099741,68cyclictest457582-21ssh11:01:480
397870997328,39cyclictest454249-21unin-run10:55:000
397870997328,28cyclictest460646-21ssh11:08:220
397870997328,28cyclictest460646-21ssh11:08:210
39787099731,26cyclictest0-21swapper/011:41:510
397870997226,30cyclictest429228-21ssh10:02:330
39787099721,16cyclictest423451-21ssh09:50:210
397870997126,41cyclictest445118-21ssh10:35:320
397870997029,25cyclictest502799-21ssh12:36:230
397870997025,14cyclictest498498-21ssh12:27:210
397870997023,16cyclictest416913-21ssh09:36:440
39787099701,29cyclictest0-21swapper/011:32:440
39787099692,62cyclictest465451-21ssh11:18:280
39787099691,64cyclictest414006-21ssh09:30:330
39787099691,64cyclictest0-21swapper/009:49:150
397870996822,3cyclictest0-21swapper/010:00:020
397870996818,7cyclictest486800-21rm12:02:580
397870996728,35cyclictest470111-21sh11:28:120
397870996724,13cyclictest404764-21ssh09:11:150
397870996724,13cyclictest404764-21ssh09:11:140
397870996722,14cyclictest475646-21ssh11:39:550
39787099671,24cyclictest0-21swapper/011:52:030
39788499662,60cyclictest403014-21munin-run08:50:013
39787699664,7cyclictest23-21ksoftirqd/112:20:031
397870996621,7cyclictest407062-21ssh09:16:080
397870996617,16cyclictest451115-21ssh10:48:220
397870996613,12cyclictest432383-21ssh10:09:210
39788499652,15cyclictest468518-21munin-run11:25:013
397884996517,43cyclictest413663-21cut09:30:013
39788499651,60cyclictest499382-21ssh12:29:263
397870996523,15cyclictest497032-21ssh12:24:370
397870996523,15cyclictest497032-21ssh12:24:370
397870996517,14cyclictest500786-21ssh12:32:070
39788499642,58cyclictest3467602-21systemd-journal10:32:043
39788499641,15cyclictest407271-21ssh09:16:343
39788499641,14cyclictest415432-21ssh09:33:493
39788499641,11cyclictest150-21pr/ttyS212:08:253
397876996424,12cyclictest23-21ksoftirqd/110:20:001
39787699641,38cyclictest0-21swapper/107:20:011
397870996454,7cyclictest12-21ksoftirqd/012:20:010
397884996328,31cyclictest432488-21sh10:09:353
397884996327,32cyclictest420773-21sed09:45:023
397884996327,32cyclictest420773-21sed09:45:023
39788499632,56cyclictest3467602-21systemd-journal11:12:383
397884996320,39cyclictest37-21ksoftirqd/310:48:213
397884996318,40cyclictest496376-21sh12:23:043
397884996318,40cyclictest496376-21sh12:23:043
39788499631,16cyclictest499929-21ssh12:30:133
397870996322,4cyclictest0-21swapper/010:15:070
39787099631,7cyclictest403234-21sshd08:51:480
397870996315,43cyclictest16850irq/134-800000009:24:430
397884996227,31cyclictest454625-21sh10:55:243
397884996227,31cyclictest454625-21sh10:55:243
39788499622,56cyclictest440188-21ssh10:25:153
39788499622,54cyclictest482868-21cron11:55:003
39788499622,20cyclictest433218-21ssh10:10:453
39788499622,15cyclictest474186-21ssh11:36:393
39788499621,8cyclictest448001-21ssh10:41:413
397870996215,11cyclictest309-21in:imuxsock08:04:590
39788499613,18cyclictest490656-21ssh12:10:523
39788499612,54cyclictest404076-21munin-run09:10:013
39788499612,54cyclictest401738-21munin-run08:25:023
39788499612,54cyclictest401738-21munin-run08:25:023
39788499612,54cyclictest398621-21munin-run07:25:023
39788499612,11cyclictest465133-21sshd11:17:493
397884996115,42cyclictest37-21ksoftirqd/309:47:523
39788499611,4cyclictest428219-21ssh10:00:083
39788499611,13cyclictest399345-21sshd07:39:423
397870996128,30cyclictest0-21swapper/008:45:420
397884996041,12cyclictest0-21swapper/311:05:003
39788499602,7cyclictest480547-21munin-run11:50:023
39788499602,7cyclictest409523-21ssh09:21:153
39788499602,6cyclictest404447-21ssh09:10:223
39788499602,6cyclictest404447-21ssh09:10:223
39788499602,5cyclictest487736-21munin-run12:05:013
39788499602,5cyclictest3467602-21systemd-journal10:21:583
39788499602,5cyclictest3467602-21systemd-journal10:21:583
39788499602,53cyclictest402773-21munin-run08:45:013
39788499601,8cyclictest436616-21ssh10:18:033
39788499601,5cyclictest492900-21ssh12:15:283
39788499601,18cyclictest503608-21ssh12:38:163
397880996036,16cyclictest3467602-21systemd-journal11:00:002
397880996036,16cyclictest3467602-21systemd-journal10:59:592
39787699607,31cyclictest401748-21kworker/1:008:40:021
39787099606,19cyclictest401187-21grep08:15:010
397870996025,11cyclictest12-21ksoftirqd/009:05:020
397870996025,11cyclictest12-21ksoftirqd/009:05:020
397870996014,13cyclictest461814-21ssh11:10:340
39788499593,51cyclictest364-21snmpd09:57:313
39788499592,5cyclictest446843-21ssh10:39:373
39788499592,51cyclictest401186-21munin-run08:15:013
39788499591,8cyclictest472237-21ssh11:32:303
39788499591,7cyclictest484894-21ssh11:59:103
39788499591,7cyclictest424424-21ssh09:52:323
39788499591,6cyclictest470848-21ssh11:29:563
39788499591,6cyclictest460476-21ssh11:08:013
39788499591,6cyclictest460476-21ssh11:08:013
39788499591,16cyclictest452310-21sh10:50:333
39788499591,11cyclictest418238-21ssh09:39:503
39788499581,5cyclictest476008-21ssh11:40:153
397880995849,6cyclictest386864-21kworker/2:309:50:022
397876995813,5cyclictest131rcu_preempt12:34:321
39787099581,53cyclictest0-21swapper/009:00:010
39788499572,4cyclictest403276-21munin-run08:55:003
39788499571,6cyclictest400915-21sshd08:09:433
39788499571,5cyclictest398082-21munin-run07:15:003
39788499571,12cyclictest364-21snmpd07:49:073
39787099572,18cyclictest404089-21grep09:10:010
397884995630,21cyclictest309-21in:imuxsock07:30:003
39788499562,5cyclictest400681-21munin-run08:05:003
397870995616,9cyclictest400149-21grep07:55:000
39788499552,49cyclictest364-21snmpd08:55:063
39788499551,5cyclictest364-21snmpd07:17:403
39787699555,11cyclictest23-21ksoftirqd/110:06:321
39787699554,11cyclictest23-21ksoftirqd/111:50:021
39787099552,20cyclictest398458-21latency_hist07:20:020
39788499542,6cyclictest364-21snmpd09:04:063
39788499542,6cyclictest364-21snmpd09:04:053
39788499542,47cyclictest364-21snmpd08:31:023
39788499541,49cyclictest364-21snmpd08:28:523
397880995446,5cyclictest398096-21kworker/2:211:12:302
39787699545,4cyclictest131rcu_preempt09:30:011
39787099542,19cyclictest401989-21unin-run08:30:010
397870995417,10cyclictest399631-21cron07:45:000
39788499532,46cyclictest401410-21cron08:16:593
39788499531,5cyclictest399646-21munin-run07:44:593
39788499531,14cyclictest397946-21cat07:10:023
397880995343,6cyclictest386864-21kworker/2:310:24:482
397880995343,6cyclictest386864-21kworker/2:310:24:472
39787699534,7cyclictest23-21ksoftirqd/110:50:021
397876995344,6cyclictest221rcuc/112:15:001
39787699533,10cyclictest23-21ksoftirqd/111:25:001
39788499521,5cyclictest364-21snmpd08:37:503
397880995246,3cyclictest386864-21kworker/2:310:08:112
397880995244,5cyclictest398096-21kworker/2:212:30:542
397880995244,5cyclictest386864-21kworker/2:310:37:522
397880995242,6cyclictest398096-21kworker/2:209:10:022
397880995242,6cyclictest386864-21kworker/2:307:10:022
39787699529,9cyclictest23-21ksoftirqd/110:00:001
397876995212,7cyclictest131rcu_preempt10:40:011
397870995230,19cyclictest16950irq/224-800000007:30:020
397870995222,11cyclictest12-21ksoftirqd/008:19:590
397870995217,7cyclictest398055-21cron07:14:590
39788499511,45cyclictest364-21snmpd07:50:313
397880995143,5cyclictest398096-21kworker/2:211:58:402
397880995143,5cyclictest386864-21kworker/2:310:28:002
397880995143,5cyclictest386864-21kworker/2:308:40:022
397880995142,5cyclictest386864-21kworker/2:310:49:022
39788499501,5cyclictest3467602-21systemd-journal07:58:083
39788499501,5cyclictest3467602-21systemd-journal07:58:083
397880995043,4cyclictest398096-21kworker/2:211:06:522
397880995043,4cyclictest398096-21kworker/2:211:06:522
397880995043,4cyclictest398096-21kworker/2:211:04:492
397880995042,5cyclictest398096-21kworker/2:211:26:592
397880995042,5cyclictest386864-21kworker/2:311:43:462
397880995041,6cyclictest398096-21kworker/2:209:56:202
397880995041,5cyclictest485640-21kworker/2:012:35:132
397880995041,5cyclictest398096-21kworker/2:209:53:132
397880995032,13cyclictest1-21systemd08:05:012
397880995028,14cyclictest275-21dbus-daemon08:00:012
397880995028,14cyclictest275-21dbus-daemon08:00:002
39787699509,7cyclictest23-21ksoftirqd/112:39:001
39787699508,9cyclictest23-21ksoftirqd/111:14:471
39787699508,10cyclictest23-21ksoftirqd/111:33:111
39787699505,9cyclictest23-21ksoftirqd/111:03:181
39787699503,11cyclictest23-21ksoftirqd/111:15:021
39787699502,10cyclictest23-21ksoftirqd/110:10:461
397876995014,6cyclictest131rcu_preempt09:36:281
39787099501,6cyclictest3467562-21sshd07:47:430
397870995014,8cyclictest397934-21cat07:10:020
397880994942,4cyclictest386864-21kworker/2:309:40:482
397880994942,4cyclictest386864-21kworker/2:309:40:482
397880994941,5cyclictest485640-21kworker/2:012:15:082
397880994941,5cyclictest398096-21kworker/2:209:25:202
397880994941,5cyclictest398096-21kworker/2:209:15:292
397880994940,6cyclictest398096-21kworker/2:209:21:042
397880994937,8cyclictest485640-21kworker/2:012:13:062
397880994924,8cyclictest386864-21kworker/2:311:50:132
39787699499,5cyclictest131rcu_preempt12:23:281
39787699499,5cyclictest131rcu_preempt12:23:281
397876994934,11cyclictest23-21ksoftirqd/112:00:001
397870994924,3cyclictest0-21swapper/007:40:010
397870994917,10cyclictest12-21ksoftirqd/007:25:010
39788499482,41cyclictest0-21swapper/307:32:093
397880994839,6cyclictest398096-21kworker/2:209:32:162
397880994839,5cyclictest398096-21kworker/2:211:20:252
39788099481,4cyclictest131rcu_preempt10:11:132
39787699489,7cyclictest23-21ksoftirqd/109:48:411
39787699489,6cyclictest131rcu_preempt12:01:131
39787699483,11cyclictest23-21ksoftirqd/112:26:561
397876994810,5cyclictest131rcu_preempt09:32:091
397870994825,19cyclictest3467718-21lldpd07:56:080
397870994825,19cyclictest3467718-21lldpd07:56:080
397880994741,4cyclictest398096-21kworker/2:212:29:442
397880994741,3cyclictest386864-21kworker/2:310:51:562
397880994739,5cyclictest398096-21kworker/2:212:21:542
397880994739,5cyclictest398096-21kworker/2:212:21:542
397880994739,5cyclictest398096-21kworker/2:211:39:272
397880994739,5cyclictest386864-21kworker/2:309:35:312
397880994738,5cyclictest398096-21kworker/2:210:04:102
397880994738,5cyclictest398096-21kworker/2:209:11:542
397880994738,5cyclictest398096-21kworker/2:209:11:532
397880994727,14cyclictest3467602-21systemd-journal11:35:012
39787699477,8cyclictest23-21ksoftirqd/110:33:291
39787699476,9cyclictest23-21ksoftirqd/111:38:571
39787699476,10cyclictest23-21ksoftirqd/111:05:291
39787699476,10cyclictest23-21ksoftirqd/111:05:291
397876994740,5cyclictest221rcuc/112:06:011
397876994737,7cyclictest23-21ksoftirqd/110:55:011
397876994736,8cyclictest23-21ksoftirqd/109:25:011
397876994735,9cyclictest23-21ksoftirqd/109:15:021
397876994735,9cyclictest23-21ksoftirqd/109:15:021
397876994719,5cyclictest131rcu_preempt08:35:011
39787699471,10cyclictest23-21ksoftirqd/111:27:341
397880994639,4cyclictest386864-21kworker/2:307:30:172
397880994638,5cyclictest386864-21kworker/2:311:49:272
397880994637,5cyclictest386864-21kworker/2:310:35:022
397880994628,13cyclictest275-21dbus-daemon10:20:012
39787699469,6cyclictest131rcu_preempt09:17:081
39787699467,5cyclictest131rcu_preempt09:50:431
39787699465,9cyclictest23-21ksoftirqd/110:05:021
397876994629,12cyclictest275-21dbus-daemon08:25:011
397876994629,12cyclictest275-21dbus-daemon08:25:001
39787699461,9cyclictest23-21ksoftirqd/110:26:521
397876994611,4cyclictest131rcu_preempt10:55:331
397876994611,4cyclictest131rcu_preempt10:55:331
397880994537,5cyclictest398096-21kworker/2:212:00:572
397880994528,10cyclictest401977-21grep08:30:002
397880994521,6cyclictest30-21ksoftirqd/211:16:542
39787699456,6cyclictest131rcu_preempt10:21:431
39787699456,6cyclictest131rcu_preempt10:21:421
397876994534,8cyclictest23-21ksoftirqd/108:05:001
397876994533,10cyclictest23-21ksoftirqd/110:42:141
397876994516,5cyclictest131rcu_preempt09:10:001
39787099451,25cyclictest0-21swapper/008:35:440
39788099444,3cyclictest131rcu_preempt12:06:202
397880994434,7cyclictest30-21ksoftirqd/207:40:012
397880994432,8cyclictest401449-21cat08:20:002
397880994429,11cyclictest30-21ksoftirqd/207:24:592
397880994428,11cyclictest30-21ksoftirqd/209:00:012
397880994422,14cyclictest275-21dbus-daemon07:30:012
397876994433,8cyclictest23-21ksoftirqd/111:52:231
39787699441,9cyclictest23-21ksoftirqd/108:15:021
397876994411,24cyclictest221rcuc/111:40:441
397880994335,5cyclictest398096-21kworker/2:208:45:012
397880994334,5cyclictest386864-21kworker/2:307:15:012
39787699432,7cyclictest23-21ksoftirqd/109:40:181
39787699432,7cyclictest23-21ksoftirqd/109:40:181
39787699431,6cyclictest150-21pr/ttyS207:33:581
39787699431,10cyclictest23-21ksoftirqd/108:45:001
39787699430,7cyclictest150-21pr/ttyS208:55:451
39787699430,6cyclictest150-21pr/ttyS207:53:401
39787099431,25cyclictest0-21swapper/008:41:510
39787699424,7cyclictest23-21ksoftirqd/108:07:511
39787699423,9cyclictest23-21ksoftirqd/108:55:011
39787699423,8cyclictest23-21ksoftirqd/107:15:021
397876994229,10cyclictest23-21ksoftirqd/107:50:011
39787699421,7cyclictest150-21pr/ttyS207:41:561
39787699420,6cyclictest150-21pr/ttyS208:49:101
39787699420,6cyclictest150-21pr/ttyS208:27:401
397870994235,3cyclictest0-21swapper/007:35:020
39787099421,2cyclictest0-21swapper/008:30:130
39787099421,20cyclictest400902-21kworker/u8:308:10:020
39787699411,6cyclictest150-21pr/ttyS207:36:291
397876994115,5cyclictest131rcu_preempt09:05:011
397876994115,5cyclictest131rcu_preempt09:05:011
397880994032,5cyclictest398096-21kworker/2:207:55:002
397880994031,6cyclictest386864-21kworker/2:308:10:012
39787699405,9cyclictest23-21ksoftirqd/108:17:001
39787099403,33cyclictest12-21ksoftirqd/008:25:020
39787099403,33cyclictest12-21ksoftirqd/008:25:010
397880993930,5cyclictest386864-21kworker/2:307:18:472
397880993830,5cyclictest398096-21kworker/2:207:48:432
397880993817,15cyclictest275-21dbus-daemon08:15:002
397880993726,8cyclictest30-21ksoftirqd/208:35:002
397880993720,10cyclictest401721-21sh08:25:012
397880993720,10cyclictest401721-21sh08:25:012
397876993721,12cyclictest400437-21systemd-run08:00:011
397876993721,12cyclictest400437-21systemd-run08:00:001
39787699369,6cyclictest131rcu_preempt07:30:031
397880993526,6cyclictest402756-21latency_hist08:45:012
397880993524,8cyclictest30-21ksoftirqd/207:45:022
39788099342,8cyclictest364-21snmpd08:53:012
39787699344,7cyclictest23-21ksoftirqd/107:10:021
39788099332,25cyclictest403855-21grep09:05:022
39788099332,25cyclictest403855-21grep09:05:012
39787699331,6cyclictest150-21pr/ttyS207:23:341
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional