You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 17:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Thu May 07, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10360995146,2cyclictest35966irq/58-eth011:50:133
10352994731,7cyclictest4169-21ssh09:50:142
10360994631,10cyclictest35966irq/58-eth009:35:133
1036099462,29cyclictest35966irq/58-eth010:50:123
10352994423,13cyclictest0-21swapper/211:40:112
1035299431,35cyclictest31393-21sh12:10:132
10360994225,13cyclictest35966irq/58-eth011:45:113
1033299423,3cyclictest131rcu_preempt07:45:110
10332994231,5cyclictest28315-21diskmemload10:45:120
1033299422,33cyclictest28315-21diskmemload10:35:130
10360993923,12cyclictest35966irq/58-eth010:15:143
1035299394,28cyclictest27508-21ssh11:50:122
1034299393,6cyclictest131rcu_preempt11:25:111
10332993918,18cyclictest12-21ksoftirqd/007:35:140
1036099382,28cyclictest35966irq/58-eth009:30:133
1035299363,25cyclictest1687-21ssh11:00:132
10360993518,11cyclictest35966irq/58-eth012:35:143
10360993515,17cyclictest35966irq/58-eth010:45:113
1034299352,23cyclictest26155-21ssh10:20:111
1034299351,26cyclictest23769-21sh11:30:121
1036099340,19cyclictest35966irq/58-eth011:55:133
1035299349,19cyclictest930-21snmpd11:08:082
1035299341,22cyclictest5418-21ssh09:15:112
1033299344,15cyclictest27528-21ssh09:45:130
1036099333,24cyclictest35966irq/58-eth010:05:153
1035299327,9cyclictest386-21ssh11:35:142
1035299325,18cyclictest27346irq/40-dwc2_hso09:40:122
10342993212,5cyclictest21-21ksoftirqd/112:20:111
1036099312,17cyclictest35966irq/58-eth008:10:123
1035299313,8cyclictest261rcuc/210:15:132
1035299312,22cyclictest0-21swapper/211:15:132
1034299312,22cyclictest12215-21sh10:46:051
10342993116,8cyclictest21-21ksoftirqd/107:10:121
1036099306,14cyclictest0-21swapper/312:10:133
1035299302,17cyclictest27346irq/40-dwc2_hso12:35:132
10352993016,11cyclictest27-21ksoftirqd/209:20:122
1033299302,19cyclictest12-21ksoftirqd/008:30:120
10332993017,7cyclictest0-21swapper/011:25:110
1033299301,21cyclictest151rcuc/011:55:120
1036099294,18cyclictest0-21swapper/310:40:133
1036099293,14cyclictest0-21swapper/309:55:133
10360992915,9cyclictest35966irq/58-eth009:20:123
1035299292,17cyclictest27346irq/40-dwc2_hso11:20:132
1034299291,24cyclictest9287-21ssh09:35:131
1035299285,14cyclictest27346irq/40-dwc2_hso07:15:112
1035299283,20cyclictest0-21swapper/210:49:062
1035299283,14cyclictest27346irq/40-dwc2_hso07:55:132
1035299281,20cyclictest31416-21ssh10:05:142
1035299281,20cyclictest13121-21ssh12:00:052
1035299281,15cyclictest27346irq/40-dwc2_hso08:45:112
1034299281,21cyclictest7976-21rm11:02:061
10332992820,5cyclictest131rcu_preempt10:50:120
10332992816,6cyclictest131rcu_preempt12:35:130
1035299274,17cyclictest27346irq/40-dwc2_hso09:00:112
1035299273,16cyclictest27346irq/40-dwc2_hso10:55:142
1035299272,19cyclictest0-21swapper/208:10:132
1033299272,15cyclictest12-21ksoftirqd/007:10:120
1036099264,16cyclictest0-21swapper/309:40:133
1035299268,14cyclictest0-21swapper/210:10:132
1035299268,14cyclictest0-21swapper/210:10:122
1035299263,17cyclictest27346irq/40-dwc2_hso08:15:122
1035299263,15cyclictest27346irq/40-dwc2_hso08:40:132
1035299262,9cyclictest3154-21ssh10:41:062
1035299262,9cyclictest14587-21ssh11:25:052
1035299262,6cyclictest28741-21kworker/u8:012:15:112
1035299261,21cyclictest26359-21ssh11:49:062
1035299261,15cyclictest4158-21ssh11:55:132
10332992618,6cyclictest12-21ksoftirqd/007:50:120
10332992610,4cyclictest131rcu_preempt09:35:300
1036099252,4cyclictest0-21swapper/310:35:133
1036099251,15cyclictest0-21swapper/309:50:143
1035299256,13cyclictest0-21swapper/209:10:132
1035299253,17cyclictest0-21swapper/210:54:062
1035299252,15cyclictest27346irq/40-dwc2_hso09:05:132
10352992511,8cyclictest0-21swapper/209:30:122
1034299254,12cyclictest28315-21diskmemload12:30:131
1034299254,12cyclictest28315-21diskmemload12:30:121
10342992512,3cyclictest28315-21diskmemload09:25:201
1036099245,12cyclictest35966irq/58-eth009:05:143
1036099242,16cyclictest0-21swapper/311:00:133
10360992416,3cyclictest0-21swapper/309:15:113
1035299248,8cyclictest13136-21apt-get09:55:122
1035299241,9cyclictest0-21swapper/208:05:152
1034299247,12cyclictest21-21ksoftirqd/112:36:081
1034299243,13cyclictest201rcuc/111:55:121
1033299243,4cyclictest12-21ksoftirqd/008:15:300
1036099234,12cyclictest0-21swapper/311:40:113
1036099232,16cyclictest0-21swapper/311:30:143
1036099232,15cyclictest0-21swapper/312:30:123
1036099232,15cyclictest0-21swapper/312:30:123
10360992314,6cyclictest0-21swapper/310:55:483
10360992313,7cyclictest0-21swapper/309:25:133
10360992313,6cyclictest0-21swapper/311:20:143
1035299232,6cyclictest0-21swapper/210:25:052
1035299232,16cyclictest0-21swapper/212:08:062
1035299232,16cyclictest0-21swapper/211:34:062
1035299232,15cyclictest0-21swapper/209:25:042
10342992317,4cyclictest10344-21latency_hist07:10:031
1034299231,17cyclictest16232-21sh11:06:051
1034299231,15cyclictest8018-21ssh10:10:141
1034299231,15cyclictest8018-21ssh10:10:131
1034299231,13cyclictest0-21swapper/109:30:211
1033299233,15cyclictest28315-21diskmemload11:30:120
1033299230,2cyclictest131rcu_preempt09:00:110
1036099221,9cyclictest0-21swapper/312:00:123
10360992215,5cyclictest0-21swapper/312:15:053
10360992214,6cyclictest0-21swapper/311:15:053
10360992213,4cyclictest0-21swapper/310:28:183
1035299224,9cyclictest22587-21if_eth110:00:142
1035299222,11cyclictest27630-21df_inode09:45:142
1035299222,11cyclictest20163-21fschecks_time11:10:142
1035299221,6cyclictest0-21swapper/212:25:142
1035299221,13cyclictest20993-21wc12:20:272
1034299221,12cyclictest19422-21memory10:50:221
1034299220,4cyclictest131rcu_preempt07:30:121
1034299220,16cyclictest19949-21ssh11:10:121
1033299221,8cyclictest25918-21latency_hist08:00:020
10332992210,3cyclictest0-21swapper/008:05:290
10360992115,4cyclictest0-21swapper/312:26:593
10360992115,4cyclictest0-21swapper/311:25:113
10360992115,4cyclictest0-21swapper/311:10:133
10360992115,4cyclictest0-21swapper/310:15:013
10360992115,4cyclictest0-21swapper/310:15:013
10360992114,5cyclictest0-21swapper/310:03:443
10360992113,6cyclictest0-21swapper/309:12:183
10360992113,5cyclictest0-21swapper/307:40:123
10360992112,6cyclictest0-21swapper/311:35:143
1036099211,14cyclictest0-21swapper/310:20:133
1036099211,12cyclictest35966irq/58-eth012:20:113
1035299216,12cyclictest26573-21grep10:20:172
1035299212,10cyclictest24818-21cut08:30:202
1035299211,9cyclictest16326-21cron07:50:012
1035299211,16cyclictest30048-21grep08:00:282
1035299211,14cyclictest27346irq/40-dwc2_hso10:30:122
10352992110,9cyclictest0-21swapper/207:10:152
10352992110,8cyclictest0-21swapper/207:30:212
10352992110,6cyclictest27346irq/40-dwc2_hso07:20:142
1034299212,16cyclictest20446-21grep07:50:271
10342992112,7cyclictest22527-21grep09:40:291
10342992112,6cyclictest26269-21timerwakeupswit12:05:291
1033299218,7cyclictest12-21ksoftirqd/008:50:020
1033299217,11cyclictest7051-21grep10:25:300
1033299216,3cyclictest131rcu_preempt11:00:250
1033299212,16cyclictest12038-21ssh10:30:130
1033299212,15cyclictest22216-21apt-get10:00:090
10332992112,6cyclictest16690-21cron12:20:010
1033299211,17cyclictest20319-21uniq07:20:160
1033299211,17cyclictest18392-21ssh09:40:130
1033299211,16cyclictest4184-21ssh09:50:140
10360992014,4cyclictest0-21swapper/309:45:133
10360992014,4cyclictest0-21swapper/308:45:023
10360992014,4cyclictest0-21swapper/307:20:013
10360992013,5cyclictest0-21swapper/307:45:153
10360992012,6cyclictest0-21swapper/308:00:133
10360992012,5cyclictest0-21swapper/311:07:043
1035299206,5cyclictest22747-21ssh10:35:202
1035299202,8cyclictest9497-21fschecks_count08:50:152
1035299202,14cyclictest9259-21ssh09:35:112
1035299201,9cyclictest8515-21kthreadcore07:40:202
1035299201,9cyclictest5257-21ssh12:30:212
1035299201,9cyclictest5257-21ssh12:30:212
1035299201,9cyclictest3882-21kthreadcore07:35:212
1035299201,9cyclictest29566-21kthreadcore08:35:212
1035299201,7cyclictest0-21swapper/207:50:262
1035299201,11cyclictest17161-21kthreadcore08:55:212
1035299201,10cyclictest19521-21kthreadcore08:25:212
1034299209,8cyclictest5563-21forks09:15:161
1034299202,3cyclictest131rcu_preempt08:50:011
1034299201,7cyclictest28315-21diskmemload10:00:191
1034299201,6cyclictest2316-21tune2fs07:35:161
1034299201,5cyclictest0-21swapper/108:05:131
1034299201,16cyclictest13214-21ssh12:00:111
1034299201,14cyclictest18506-21ssh11:45:131
1034299201,13cyclictest30220-21ssh10:40:131
10342992010,7cyclictest13209-21awk11:40:251
10342992010,2cyclictest0-21swapper/109:05:231
1033299209,9cyclictest12-21ksoftirqd/009:10:140
1033299209,8cyclictest9756-21ssh11:40:160
1033299209,8cyclictest12-21ksoftirqd/011:10:120
1033299206,4cyclictest131rcu_preempt12:00:180
1033299204,4cyclictest131rcu_preempt10:20:140
1033299203,15cyclictest23631-21sh09:25:130
1033299201,3cyclictest131rcu_preempt10:15:120
10332992013,3cyclictest12-21ksoftirqd/011:15:120
1033299201,16cyclictest31396-21ssh12:10:140
1036099193,13cyclictest0-21swapper/310:30:133
10360991914,3cyclictest0-21swapper/308:05:233
10360991913,4cyclictest0-21swapper/312:05:233
10360991913,4cyclictest0-21swapper/309:00:123
10360991912,5cyclictest0-21swapper/307:35:283
1035299192,8cyclictest25165-21wc07:25:162
1035299191,9cyclictest14065-21kthreadcore08:20:182
1034299199,7cyclictest30779-21kthreadcore09:10:201
1034299197,4cyclictest131rcu_preempt11:35:201
1034299192,15cyclictest22315-21kthreadcore07:55:191
1034299192,11cyclictest27531-21diskmemload09:45:131
1034299191,8cyclictest27795-21kthreadcore07:25:211
10342991914,3cyclictest16320-21sshd07:50:011
10342991911,5cyclictest23195-21apt-config09:25:011
10342991910,7cyclictest11650-21cron10:30:011
10342991910,6cyclictest31919-21cron08:40:011
1034299190,17cyclictest29847-21timerandwakeup08:00:271
1033299197,9cyclictest31652-21grep11:50:290
1033299195,10cyclictest12-21ksoftirqd/007:40:130
1033299192,8cyclictest0-21swapper/010:42:080
1033299192,3cyclictest131rcu_preempt09:00:010
1033299192,14cyclictest22272-21ssh12:05:130
1033299192,14cyclictest15305-21sshd07:15:120
1033299192,13cyclictest13146-21ssh09:55:120
10332991911,6cyclictest16235-21sshd11:06:050
1033299191,15cyclictest25043-21ssh10:55:130
1033299191,14cyclictest23751-21apt-get09:05:130
10332991910,6cyclictest27435-21cat12:25:200
1036099184,10cyclictest35966irq/58-eth007:25:143
1036099182,10cyclictest0-21swapper/308:50:143
1036099182,10cyclictest0-21swapper/308:30:133
10360991813,4cyclictest0-21swapper/307:50:263
10360991812,4cyclictest0-21swapper/308:45:263
10360991812,4cyclictest0-21swapper/308:15:193
10360991812,4cyclictest0-21swapper/307:30:133
10360991811,5cyclictest0-21swapper/308:35:253
1036099181,12cyclictest0-21swapper/308:20:263
1036099181,11cyclictest0-21swapper/308:55:133
1034299189,6cyclictest6416-21kthreadcore11:20:201
1034299185,10cyclictest12761-21cron09:55:011
1034299182,8cyclictest25801-21grep10:20:011
1034299181,9cyclictest14483-21cut08:20:191
1034299181,8cyclictest131rcu_preempt12:25:211
1034299181,7cyclictest30911-21apt-get12:10:041
10342991813,3cyclictest0-21swapper/108:25:281
10342991812,3cyclictest21394-21if_err_eth110:35:161
10342991811,2cyclictest9650irq/52-mmc007:40:151
10342991810,5cyclictest32590-21needreboot11:15:231
1034299180,9cyclictest28681-21ntp_kernel_pll_10:55:251
1033299189,7cyclictest32175-21apt-get08:40:120
1033299183,7cyclictest131rcu_preempt08:00:110
1033299183,4cyclictest4350irq/33-fe00b88010:10:030
1033299183,4cyclictest4350irq/33-fe00b88010:10:030
1033299182,12cyclictest151rcuc/009:20:110
1033299181,8cyclictest22498-21latency_hist08:30:030
1033299181,4cyclictest131rcu_preempt08:10:180
10332991813,3cyclictest4350irq/33-fe00b88008:20:200
1033299181,2cyclictest131rcu_preempt11:45:160
1033299181,2cyclictest131rcu_preempt11:35:180
1033299181,2cyclictest131rcu_preempt09:15:150
1033299181,14cyclictest17110-21ssh12:20:120
10332991810,5cyclictest6518-21mailstats12:30:270
10332991810,5cyclictest6518-21mailstats12:30:270
1033299180,6cyclictest1925-21tr09:30:190
10360991712,3cyclictest0-21swapper/307:10:183
10360991711,4cyclictest0-21swapper/308:25:143
10360991711,4cyclictest0-21swapper/307:55:163
10360991711,4cyclictest0-21swapper/307:20:293
1035299171,7cyclictest10444-21latency_hist07:10:022
1034299176,8cyclictest20160-21apt-get07:20:131
1034299172,2cyclictest131rcu_preempt11:50:201
1034299172,10cyclictest0-21swapper/108:15:121
1034299171,7cyclictest0-21swapper/112:15:211
10342991712,3cyclictest0-21swapper/108:30:281
10342991711,4cyclictest6893-21processes08:10:261
1034299170,5cyclictest13628-21cut09:55:171
1034299170,12cyclictest31412-21ssh10:05:141
1033299178,7cyclictest3584-21cat10:05:320
1033299174,10cyclictest27487-21kthreadcore07:25:210
10332991710,5cyclictest28315-21diskmemload11:20:190
1034299162,9cyclictest0-21swapper/108:55:131
1034299162,9cyclictest0-21swapper/108:40:131
1034299162,11cyclictest15532-21idleruntime07:15:151
1034299161,7cyclictest131rcu_preempt10:30:191
1034299161,11cyclictest22775-21grep09:00:271
10342991610,4cyclictest9561-21hwlatdetect08:50:161
1033299161,3cyclictest131rcu_preempt07:30:130
1033299161,12cyclictest27372-21apt-get08:35:120
10332991610,4cyclictest12-21ksoftirqd/008:50:300
1036099141,10cyclictest0-21swapper/307:10:023
1033299142,9cyclictest10271-21latency_hist07:10:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional