You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-24 - 15:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Wed Jun 24, 2026 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2991999705,54cyclictest17890-21ssh12:30:120
29919995717,34cyclictest8088-21ssh11:30:120
2993499543,40cyclictest20275-21ssh11:55:121
2991999545,35cyclictest25903-21ssh11:40:120
2991999534,37cyclictest5292-21ssh09:40:110
2991999522,36cyclictest8768-21ssh10:00:120
29934994935,10cyclictest14655-21apt-get12:10:121
29934994935,10cyclictest14655-21apt-get12:10:121
2994399481,42cyclictest15555-21ssh10:40:122
2991999472,34cyclictest1619-21ssh09:20:130
29934994536,6cyclictest29906-21cyclictest10:30:141
29919994522,10cyclictest12-21ksoftirqd/012:35:110
2995499443,25cyclictest35966irq/58-eth011:25:113
29934994437,3cyclictest3744-21apt-get07:50:111
2991999441,32cyclictest25124-21ssh09:15:150
29954994319,18cyclictest0-21swapper/311:32:433
29934994335,4cyclictest28681-21apt-get09:35:141
29934994333,4cyclictest1027-21apt-get10:50:131
29934994333,4cyclictest1027-21apt-get10:50:131
29934994326,12cyclictest6582-21ssh10:35:121
2993499432,29cyclictest17758-21ssh10:05:141
2994399421,31cyclictest20265-21ssh11:55:112
29934994226,11cyclictest26843-21apt-get12:35:111
29919994224,5cyclictest12-21ksoftirqd/011:05:140
29934994134,3cyclictest10036-21apt-get10:55:111
29934994130,7cyclictest2480-21apt-get07:15:121
29934994115,11cyclictest21081-21ssh10:25:071
2991999401,32cyclictest1054-21ssh10:50:140
2991999401,32cyclictest1054-21ssh10:50:140
2995499392,24cyclictest35966irq/58-eth012:00:133
29934993931,5cyclictest8757-21apt-get10:00:121
29934993926,8cyclictest0-21swapper/111:20:111
29934993918,7cyclictest13611-21ssh12:09:111
29934993915,10cyclictest6116-21ssh11:28:061
29934993913,12cyclictest14901-21ssh11:50:231
29934993912,10cyclictest25008-21kthreadcore12:15:191
2995499385,20cyclictest35966irq/58-eth009:35:153
29934993829,3cyclictest31358-21apt-get07:45:131
29934993826,8cyclictest23273-21apt-get09:50:141
29934993824,10cyclictest12072-21apt-get07:25:111
29934993815,10cyclictest20856-21rm09:47:401
29934993812,11cyclictest20685-21rm10:24:531
29934993812,10cyclictest2025-21ssh10:13:561
2991999382,27cyclictest14260-21ssh09:45:120
2994399372,19cyclictest23606-21ssh11:37:432
29934993729,5cyclictest29100-21apt-get08:50:111
29934993729,5cyclictest27840-21apt-get08:15:141
29934993715,8cyclictest9477-21sh10:17:121
29934993715,8cyclictest20649-21ssh11:18:101
29934993714,15cyclictest15655-21diskmemload11:37:351
29934993712,9cyclictest7523-21ssh09:21:491
2995499361,22cyclictest35966irq/58-eth012:15:123
2994399366,19cyclictest27-21ksoftirqd/207:10:132
29934993628,3cyclictest29285-21apt-get12:00:141
29934993615,8cyclictest10551-21ssh11:11:461
29934993615,7cyclictest15655-21diskmemload09:30:431
29934993614,8cyclictest27811-21grep10:45:221
29934993613,8cyclictest27188-21ssh11:04:231
29934993613,7cyclictest0-21swapper/110:44:191
29934993612,9cyclictest6107-21ssh12:21:591
29934993612,8cyclictest14749-21ssh12:26:381
29934993610,9cyclictest21967-21ssh09:11:411
29934993514,7cyclictest24983-21ssh09:15:041
29934993513,9cyclictest15655-21diskmemload11:06:131
29934993513,7cyclictest15655-21diskmemload11:41:251
29934993512,9cyclictest18563-21cut12:30:181
2995499343,17cyclictest35966irq/58-eth010:50:123
2995499343,17cyclictest35966irq/58-eth010:50:123
2993499342,27cyclictest32257-21ssh09:55:141
29934993418,13cyclictest0-21swapper/108:35:131
29934993416,14cyclictest0-21swapper/111:30:121
29934993413,9cyclictest10920-21sh09:41:291
29934993413,8cyclictest8798-21ssh11:47:251
29943993318,9cyclictest27-21ksoftirqd/212:35:112
29934993319,11cyclictest0-21swapper/108:45:131
29934993319,11cyclictest0-21swapper/107:20:111
29934993318,11cyclictest0-21swapper/108:30:111
2995499322,21cyclictest35966irq/58-eth010:00:123
29943993223,3cyclictest26782-21ssh10:10:122
29934993219,8cyclictest0-21swapper/109:25:101
29934993210,19cyclictest12610-21kthreadcore09:05:211
2991999325,17cyclictest12-21ksoftirqd/012:15:110
2991999321,22cyclictest0-21swapper/011:50:110
2994399315,16cyclictest27346irq/40-dwc2_hso07:20:112
29943993113,4cyclictest27-21ksoftirqd/209:05:252
2993499319,8cyclictest0-21swapper/107:35:251
29934993119,9cyclictest0-21swapper/107:10:131
2994399308,3cyclictest131rcu_preempt08:40:012
2994399303,18cyclictest27346irq/40-dwc2_hso07:55:102
29934993020,7cyclictest22888-21ntp_192.168.11408:40:231
29934993018,8cyclictest27525-21apt-get08:15:011
29934993018,8cyclictest27525-21apt-get08:15:011
29934993017,9cyclictest18416-21if_err_eth008:05:181
29934993017,11cyclictest5010-21ps08:55:251
29934993016,10cyclictest0-21swapper/107:55:101
29934993014,13cyclictest6569-21kthreadcore08:25:211
29934993013,6cyclictest3635-21grep08:20:231
29934993013,6cyclictest3635-21grep08:20:221
2991999302,22cyclictest3492-21ssh12:02:430
2995499297,17cyclictest0-21swapper/311:10:433
2995499293,16cyclictest35966irq/58-eth010:45:123
2994399292,11cyclictest32519-21ssh12:20:142
29943992910,7cyclictest27-21ksoftirqd/209:20:222
29934992917,9cyclictest29978-21date07:10:011
29934992917,9cyclictest29978-21date07:10:011
2991999292,21cyclictest13227-21ssh12:08:430
29919992911,13cyclictest2871-1kworker/u9:011:39:430
29919992911,13cyclictest25302-1kworker/u9:211:13:270
29954992818,6cyclictest35966irq/58-eth012:20:123
2994399288,13cyclictest27-21ksoftirqd/211:33:152
2994399285,14cyclictest27346irq/40-dwc2_hso10:35:122
2994399284,14cyclictest27346irq/40-dwc2_hso09:30:122
2994399283,20cyclictest0-21swapper/211:47:432
29943992822,4cyclictest27-21ksoftirqd/212:00:182
29934992816,9cyclictest13674-21ps08:00:171
29934992816,8cyclictest28607-21kthreadcore07:40:221
29934992816,10cyclictest0-21swapper/107:30:111
29934992816,10cyclictest0-21swapper/107:30:111
2991999284,13cyclictest23272-21ssh09:50:120
2995499272,16cyclictest35966irq/58-eth011:40:133
2994399273,9cyclictest0-21swapper/212:12:432
2994399273,9cyclictest0-21swapper/212:12:432
2994399272,18cyclictest0-21swapper/208:00:142
2994399272,14cyclictest0-21swapper/209:00:112
2994399271,20cyclictest23300-21ssh09:50:142
29934992712,11cyclictest6774-21irqcore09:00:161
29954992611,10cyclictest35966irq/58-eth012:05:153
2994399263,13cyclictest27346irq/40-dwc2_hso07:30:122
2994399263,13cyclictest27346irq/40-dwc2_hso07:30:122
2994399262,12cyclictest1038-21ssh10:50:132
2994399262,12cyclictest1038-21ssh10:50:132
2994399262,12cyclictest1005366ssh10:55:112
29943992617,5cyclictest27346irq/40-dwc2_hso10:45:112
29943992612,8cyclictest27-21ksoftirqd/209:12:192
29943992612,4cyclictest27-21ksoftirqd/208:20:232
29943992612,4cyclictest27-21ksoftirqd/208:20:232
2994399261,15cyclictest930-21snmpd09:15:252
2994399261,15cyclictest3297-21ssh10:15:132
2991999262,17cyclictest2412-21ssh11:45:130
29919992618,5cyclictest15655-21diskmemload09:10:130
29919992611,8cyclictest12-21ksoftirqd/007:50:120
29954992512,9cyclictest35966irq/58-eth012:30:143
2994399259,5cyclictest27-21ksoftirqd/212:16:062
2994399258,4cyclictest27-21ksoftirqd/212:30:542
2994399258,4cyclictest27-21ksoftirqd/209:58:232
2994399257,8cyclictest27-21ksoftirqd/212:06:422
2994399256,6cyclictest131rcu_preempt08:05:272
2994399256,11cyclictest27-21ksoftirqd/209:25:152
2994399253,15cyclictest27-21ksoftirqd/208:45:012
2994399252,19cyclictest0-21swapper/210:30:142
2994399252,18cyclictest0-21swapper/211:05:142
2994399251,9cyclictest11913-21ssh11:13:272
29943992517,5cyclictest15655-21diskmemload11:25:222
2994399251,19cyclictest11484-21grep11:50:152
2991999253,3cyclictest131rcu_preempt10:35:160
2991999251,19cyclictest30134-21ssh10:30:140
2991999251,17cyclictest15655-21diskmemload09:25:110
29919992510,7cyclictest29906-21cyclictest12:10:120
29919992510,7cyclictest29906-21cyclictest12:10:120
2991999250,19cyclictest21168-21ssh10:25:120
2995499242,18cyclictest35966irq/58-eth011:20:123
2994399247,5cyclictest27-21ksoftirqd/210:20:122
2994399247,4cyclictest27-21ksoftirqd/211:43:102
2994399247,11cyclictest27-21ksoftirqd/210:28:362
2994399246,7cyclictest27-21ksoftirqd/212:28:532
2994399246,7cyclictest27-21ksoftirqd/211:15:122
2994399243,12cyclictest0-21swapper/208:25:132
29943992416,5cyclictest9654-21ssh10:00:192
2994399241,20cyclictest26142-21ls11:20:222
2994399241,14cyclictest14281-21sh09:45:132
29943992410,8cyclictest27-21ksoftirqd/210:07:272
2991999243,18cyclictest21871-21timerandwakeup08:05:290
2991999241,17cyclictest31591-21sh11:25:130
2995499235,16cyclictest33-21ksoftirqd/309:30:133
2995499232,12cyclictest0-21swapper/309:45:133
29954992312,7cyclictest35966irq/58-eth009:40:133
2995499231,16cyclictest35966irq/58-eth010:10:123
2995499231,14cyclictest35966irq/58-eth011:05:133
2994399238,4cyclictest27-21ksoftirqd/211:00:282
2994399235,4cyclictest27-21ksoftirqd/209:37:002
29943992310,4cyclictest27-21ksoftirqd/209:44:052
2991999238,6cyclictest13180-21cut10:55:190
2991999233,16cyclictest151rcuc/009:35:140
2991999231,18cyclictest26940-21cut10:10:150
2991999231,17cyclictest2662-21cut07:15:150
2995499223,14cyclictest0-21swapper/310:58:243
2995499222,10cyclictest0-21swapper/311:37:433
29954992212,7cyclictest35966irq/58-eth009:55:143
2995499221,15cyclictest0-21swapper/312:35:123
2994399228,8cyclictest27-21ksoftirqd/208:10:182
2994399228,8cyclictest27-21ksoftirqd/208:10:182
2994399224,5cyclictest27-21ksoftirqd/208:50:282
2994399222,12cyclictest27346irq/40-dwc2_hso07:35:132
2994399221,9cyclictest27-21ksoftirqd/207:40:262
2991999223,13cyclictest0-21swapper/007:25:110
2991999223,12cyclictest0-21swapper/009:05:140
2991999221,6cyclictest53646usb-storage08:15:040
2991999221,16cyclictest12217-21ssh10:20:130
2995499219,6cyclictest0-21swapper/310:27:443
2995499218,10cyclictest0-21swapper/309:10:303
2995499212,11cyclictest0-21swapper/310:40:123
2994399212,10cyclictest3943-21tune2fs07:50:152
2994399211,8cyclictest0-21swapper/207:15:122
2991999219,9cyclictest31212-21cron11:25:010
2991999219,4cyclictest131rcu_preempt11:15:300
2991999211,9cyclictest32607-21df12:20:140
2991999211,14cyclictest9168-21cut12:25:150
2991999211,14cyclictest3509-21cut10:15:150
2991999210,15cyclictest4824-21latency_hist08:25:010
2991999210,15cyclictest4824-21latency_hist08:25:010
2995499204,13cyclictest35966irq/58-eth007:10:123
2995499204,10cyclictest0-21swapper/311:45:133
2995499203,11cyclictest0-21swapper/311:00:143
2995499202,6cyclictest0-21swapper/311:19:433
2995499202,14cyclictest35966irq/58-eth007:35:123
29954992011,6cyclictest35966irq/58-eth009:50:123
2994399209,4cyclictest27-21ksoftirqd/208:45:172
29943992010,4cyclictest27-21ksoftirqd/208:15:142
2991999201,7cyclictest3470-21ntp_states09:55:260
2991999201,12cyclictest6048-21latency_hist09:00:020
2991999200,13cyclictest27526-21latency_hist08:15:010
2991999200,13cyclictest27526-21latency_hist08:15:010
2995499199,8cyclictest0-21swapper/309:26:103
2995499199,7cyclictest35966irq/58-eth009:15:133
2995499192,11cyclictest0-21swapper/310:05:153
29954991914,3cyclictest33-21ksoftirqd/312:30:023
29954991914,3cyclictest33-21ksoftirqd/308:45:293
2995499191,12cyclictest0-21swapper/309:05:173
2995499191,12cyclictest0-21swapper/308:55:213
2994399193,12cyclictest27346irq/40-dwc2_hso08:55:122
2994399191,9cyclictest9997-21cut08:30:152
2994399191,9cyclictest30019-21latency_hist07:10:012
2994399191,9cyclictest30019-21latency_hist07:10:012
2994399191,9cyclictest15360-21missed_timers07:25:212
2994399191,10cyclictest31818-21irqcore07:45:172
2991999197,3cyclictest131rcu_preempt07:55:130
2991999196,10cyclictest18371-21kthreadcore10:40:210
2991999191,8cyclictest2762-21tail07:45:270
29919991912,4cyclictest21286-21kthreadcore11:55:190
29919991910,6cyclictest0-21swapper/007:30:170
29919991910,6cyclictest0-21swapper/007:30:170
2995499189,7cyclictest0-21swapper/310:22:183
2995499185,8cyclictest0-21swapper/310:15:133
2995499182,11cyclictest0-21swapper/312:12:433
2995499182,11cyclictest0-21swapper/312:12:433
2995499182,10cyclictest0-21swapper/308:50:133
2995499182,10cyclictest0-21swapper/308:40:133
2995499182,10cyclictest0-21swapper/307:25:133
29954991812,4cyclictest0-21swapper/311:50:123
2995499181,13cyclictest35966irq/58-eth007:20:123
2995499181,10cyclictest35966irq/58-eth011:55:113
29954991810,6cyclictest0-21swapper/309:20:053
2991999183,10cyclictest0-21swapper/011:03:240
2991999181,8cyclictest19843-21fschecks_count09:30:160
2991999181,7cyclictest22587-21users10:05:320
2991999181,7cyclictest12890-21kthreadcore08:30:210
29919991810,5cyclictest21987-21irqcore07:35:180
29919991810,5cyclictest17219-21grep08:00:290
29919991810,5cyclictest0-21swapper/008:25:150
2995499177,8cyclictest0-21swapper/310:35:213
2995499177,8cyclictest0-21swapper/310:30:083
2995499172,9cyclictest0-21swapper/308:10:143
2995499172,9cyclictest0-21swapper/308:10:143
2995499172,10cyclictest0-21swapper/309:00:123
2995499172,10cyclictest0-21swapper/308:20:133
2995499172,10cyclictest0-21swapper/308:20:133
2995499172,10cyclictest0-21swapper/308:15:143
2995499172,10cyclictest0-21swapper/308:00:143
2995499172,10cyclictest0-21swapper/307:15:133
29954991712,3cyclictest33-21ksoftirqd/307:50:173
2995499171,13cyclictest0-21swapper/307:30:133
2995499171,13cyclictest0-21swapper/307:30:133
2995499171,10cyclictest0-21swapper/308:25:243
2991999172,12cyclictest6291-21acpi09:00:100
2991999172,12cyclictest32518-21ntp_192.168.11408:50:240
2991999172,11cyclictest27261-21kthreadcore07:40:200
2991999171,6cyclictest9344-21kthreadcore07:20:200
2991999171,11cyclictest0-21swapper/010:45:130
2995499162,10cyclictest0-21swapper/308:30:123
2995499162,10cyclictest0-21swapper/307:45:133
2995499161,12cyclictest0-21swapper/307:40:233
2991999161,7cyclictest14820-21fschecks_time08:35:170
2991999161,5cyclictest0-21swapper/008:40:220
2991999161,12cyclictest28034-21cat08:45:280
2991999161,11cyclictest30097-21apt-get07:10:120
2995499151,10cyclictest0-21swapper/307:55:113
2995499141,11cyclictest0-21swapper/308:35:243
2991999132,8cyclictest30008-21latency_hist07:10:010
2991999132,8cyclictest30008-21latency_hist07:10:010
2995499122,7cyclictest0-21swapper/308:05:153
2995499111,7cyclictest0-21swapper/307:10:013
2995499111,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional