You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-06 - 07:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Tue Jan 06, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13931991066,62cyclictest0-21swapper/021:30:260
13931999143,15cyclictest0-21swapper/021:15:280
1393199906,47cyclictest0-21swapper/023:30:270
179082810,4sleep228544-21kworker/u8:222:20:132
13931997825,35cyclictest26499-21idleruntime-cro19:45:140
1393199777,56cyclictest5150-21sh21:45:130
1393199745,59cyclictest9-21ksoftirqd/023:20:400
112342740,5sleep21393399cyclictest20:30:332
1393199722,57cyclictest17643-21grep23:45:410
1393199721,58cyclictest21828-21basename22:30:270
134362720,4sleep20-21swapper/219:07:272
1393199711,54cyclictest19839-21cron23:55:130
1393199691,6cyclictest18192-21wget22:20:270
1393299687,28cyclictest0-21swapper/122:00:271
1393199683,13cyclictest0-21swapper/019:10:170
1393199682,53cyclictest8638-21idleruntime-cro20:25:140
1393199681,52cyclictest7245-21aten_rbpower_en21:50:290
1393199681,51cyclictest0-21swapper/023:40:310
1393199681,51cyclictest0-21swapper/023:40:310
13934996637,23cyclictest1125-21runrttasks19:10:173
13934996623,9cyclictest29995-21ls22:50:403
13934996623,9cyclictest29995-21ls22:50:403
13933996552,7cyclictest0-21swapper/219:10:162
1393199654,47cyclictest0-21swapper/019:20:250
1393199654,47cyclictest0-21swapper/019:20:250
1393199651,50cyclictest30936-21wget22:55:290
13931996511,12cyclictest7750irq/56-dwc_otg22:05:240
1393199650,49cyclictest14517-21latency_hist23:40:150
1393299643,30cyclictest0-21swapper/100:15:281
1393299643,30cyclictest0-21swapper/100:15:281
1393199643,48cyclictest466-21in:imuxsock19:26:240
1393199641,48cyclictest31330-21fschecks_count00:25:320
1393199641,41cyclictest0-21swapper/021:10:140
1393499627,25cyclictest0-21swapper/321:15:283
1393199622,5cyclictest0-21swapper/019:30:310
1393199621,52cyclictest0-21swapper/021:15:140
1393199621,42cyclictest25401-21basename00:10:140
1393499617,9cyclictest7950irq/56-dwc_otg_23:55:263
1393199611,47cyclictest0-21swapper/023:25:250
1393199601,50cyclictest32364-21cut21:30:150
1393199597,39cyclictest0-21swapper/022:00:200
1393199591,45cyclictest0-21swapper/019:49:350
1393199588,13cyclictest0-21swapper/020:35:270
1393199582,40cyclictest0-21swapper/000:00:360
1393199581,47cyclictest0-21swapper/019:56:550
1393199581,47cyclictest0-21swapper/019:56:550
1393199581,44cyclictest0-21swapper/023:55:320
1393199581,42cyclictest3315-21seq21:40:060
1393199581,42cyclictest3315-21seq21:40:060
1393199580,43cyclictest21315-21chrt22:27:400
13934995722,19cyclictest25526-21wc00:10:153
1393199572,46cyclictest0-21swapper/021:20:270
1393199572,44cyclictest0-21swapper/019:35:360
1393199571,8cyclictest16111-21munin-run22:15:130
1393199571,39cyclictest5914-21grep20:15:370
1393199571,13cyclictest28916-21latency_hist22:50:150
1393199570,47cyclictest15619-21taskset20:42:180
1393199561,41cyclictest0-21swapper/021:02:200
1393199560,42cyclictest0-21swapper/022:50:380
1393199560,42cyclictest0-21swapper/022:50:380
1393199552,50cyclictest32381-21aten_rbpower_vo20:00:290
1393199552,43cyclictest0-21swapper/023:05:380
1393199551,36cyclictest26378-21grep00:10:360
1393199551,14cyclictest10455-21idleruntime-cro20:30:140
1393199550,47cyclictest21073-21runrttasks20:57:380
1393199550,45cyclictest28993-21chrt00:20:060
1393199550,45cyclictest28993-21chrt00:20:060
1393199550,39cyclictest30621-21taskset00:22:520
1393499549,29cyclictest22882-21runrttasks21:02:443
13934995415,12cyclictest2758-21apt-config00:40:143
1393399547,9cyclictest7850irq/56-dwc_otg_20:20:272
1393299544,8cyclictest0-21swapper/119:10:171
13931995436,5cyclictest7850irq/56-dwc_otg_23:10:410
1393199542,49cyclictest1894-21fschecks_count20:05:310
1393199541,13cyclictest10144-21grep21:55:420
1393199540,39cyclictest7067-21chrt23:18:520
1393199540,34cyclictest0-21swapper/020:52:170
263962530,2sleep10-21swapper/122:40:411
1393499536,28cyclictest0-21swapper/319:25:193
13934995320,8cyclictest26377-21ps00:10:363
1393199533,35cyclictest0-21swapper/020:10:340
1393199531,40cyclictest0-21swapper/020:47:370
1393199531,40cyclictest0-21swapper/020:47:370
1393499524,6cyclictest7950irq/56-dwc_otg_19:55:263
1393499524,6cyclictest7950irq/56-dwc_otg_19:55:263
13934995216,21cyclictest7950irq/56-dwc_otg_23:10:303
1393199529,5cyclictest25673-21fschecks_count22:40:320
1393199520,35cyclictest973-21latency_hist00:35:140
13934995117,7cyclictest21781-21apt-get22:30:263
1393199511,36cyclictest0-21swapper/021:45:310
1393199511,30cyclictest0-21swapper/022:35:280
1393199511,28cyclictest485-21perl23:00:290
13934995016,10cyclictest7950irq/56-dwc_otg_23:10:143
1393199502,19cyclictest0-21swapper/019:15:410
1393199501,33cyclictest0-21swapper/000:39:440
1393199500,40cyclictest0-21swapper/019:54:210
1393499497,23cyclictest0-21swapper/320:10:293
13934994924,7cyclictest6085-21ntp_states21:45:353
13934994922,11cyclictest15481-21/usr/sbin/munin23:40:393
13934994922,11cyclictest15481-21/usr/sbin/munin23:40:393
1393399494,24cyclictest0-21swapper/220:10:262
1393299497,27cyclictest0-21swapper/123:00:251
1393299497,27cyclictest0-21swapper/119:40:271
1393299492,23cyclictest0-21swapper/119:50:231
1393499487,24cyclictest0-21swapper/322:10:273
13934994818,14cyclictest7950irq/56-dwc_otg_21:40:143
13934994818,14cyclictest7950irq/56-dwc_otg_21:40:143
13934994814,10cyclictest7950irq/56-dwc_otg_21:50:403
1393399487,26cyclictest0-21swapper/200:00:262
1393299487,25cyclictest0-21swapper/100:25:291
1393399477,25cyclictest0-21swapper/223:05:272
1393299477,26cyclictest0-21swapper/120:05:281
1393299477,23cyclictest0-21swapper/120:40:281
1393299471,5cyclictest18066-21ls22:20:141
13932994710,13cyclictest32256-21sed21:30:151
1393199470,36cyclictest12223-21taskset20:35:100
1393199470,33cyclictest1125-21runrttasks22:16:450
13934994619,12cyclictest19807-21latency_hist22:25:143
1393399467,25cyclictest0-21swapper/223:55:272
1393399467,24cyclictest0-21swapper/219:55:192
1393399467,24cyclictest0-21swapper/219:55:192
1393299467,23cyclictest0-21swapper/123:45:271
1393299466,27cyclictest0-21swapper/119:55:201
1393299466,27cyclictest0-21swapper/119:55:201
1393499457,26cyclictest0-21swapper/320:30:263
1393499457,24cyclictest0-21swapper/322:45:263
1393499456,24cyclictest0-21swapper/319:50:253
13934994521,7cyclictest9240-21apt-get23:25:253
13934994516,9cyclictest29013-21modprobe00:20:143
13934994516,9cyclictest29013-21modprobe00:20:143
13934994513,10cyclictest7950irq/56-dwc_otg_00:30:143
1393299458,23cyclictest0-21swapper/119:25:191
1393299457,25cyclictest0-21swapper/123:35:271
1393299456,17cyclictest15235-21/usr/sbin/munin23:40:341
1393299456,17cyclictest15235-21/usr/sbin/munin23:40:341
13932994511,9cyclictest0-21swapper/100:05:251
1393299451,18cyclictest7850irq/56-dwc_otg_22:35:281
136012453,35sleep00-21swapper/019:09:370
163132440,19sleep27950irq/56-dwc_otg_22:15:142
1393499447,23cyclictest0-21swapper/322:55:283
1393499446,24cyclictest0-21swapper/323:45:273
13934994422,12cyclictest8963-21chrt23:24:593
13934994416,15cyclictest468-21rs:main3
13934994416,10cyclictest7950irq/56-dwc_otg_23:34:163
1393399447,25cyclictest0-21swapper/223:10:262
1393399443,28cyclictest0-21swapper/220:55:252
1393399441,22cyclictest7950irq/56-dwc_otg_21:25:422
1393299449,8cyclictest21838-21wget22:30:271
1393299447,10cyclictest7950irq/56-dwc_otg_00:10:241
1393299445,15cyclictest32064-21runrttasks22:56:181
1393499437,23cyclictest0-21swapper/320:40:283
1393499437,22cyclictest0-21swapper/319:20:273
1393499437,22cyclictest0-21swapper/319:20:273
13934994328,12cyclictest12475-21/usr/sbin/munin20:35:383
13934994319,11cyclictest7108-21chrt23:19:233
13934994316,11cyclictest15885-21cpu19:15:303
1393399437,24cyclictest0-21swapper/200:05:282
1393399437,23cyclictest0-21swapper/223:50:272
1393399432,26cyclictest7950irq/56-dwc_otg_00:35:292
1393299436,10cyclictest20699-21wc23:55:361
1393299433,10cyclictest0-21swapper/120:50:271
13932994311,13cyclictest5771-21/usr/sbin/munin21:45:311
13934994227,12cyclictest16900-21ls22:15:343
13934994220,10cyclictest22790-21chrt19:34:273
13934994216,12cyclictest26144-21ps22:40:393
13934994214,10cyclictest7950irq/56-dwc_otg_21:27:323
1393299427,23cyclictest0-21swapper/123:15:281
1393299426,24cyclictest0-21swapper/120:15:281
1393299424,22cyclictest4106-21fschecks_time23:10:301
1393499419,12cyclictest7950irq/56-dwc_otg_21:32:383
1393499417,21cyclictest0-21swapper/321:20:283
13934994121,10cyclictest26515-21sshd21:12:453
13934994117,10cyclictest12160-21chrt22:01:233
13933994115,5cyclictest26966-21fschecks_count19:45:312
1393299417,25cyclictest0-21swapper/119:45:271
1393299416,23cyclictest0-21swapper/120:10:291
1393299412,31cyclictest0-21swapper/122:23:171
1393499406,22cyclictest0-21swapper/323:00:253
13934994015,9cyclictest3585-21apt-get21:40:263
1393299408,11cyclictest7950irq/56-dwc_otg_21:05:141
1393299407,12cyclictest7193-21/usr/sbin/munin21:50:331
1393299402,23cyclictest7750irq/56-dwc_otg21:25:141
1393299402,11cyclictest0-21swapper/120:30:241
1393299401,25cyclictest0-21swapper/122:05:261
65592390,2sleep30-21swapper/320:17:143
1393499397,21cyclictest0-21swapper/319:40:273
1393499393,5cyclictest0-21swapper/300:00:243
13934993926,11cyclictest13542-21expr22:05:383
13934993924,12cyclictest2750-21unixbench_multi20:05:423
13934993923,12cyclictest10146-21head21:55:423
13934993916,9cyclictest21291-21sh21:00:133
13934993915,10cyclictest1099-21sshd20:01:563
13934993913,9cyclictest0-21swapper/322:25:193
13934993911,18cyclictest19269-21sshd20:52:433
1393399397,22cyclictest0-21swapper/221:35:272
1393399397,22cyclictest0-21swapper/221:35:272
1393299393,7cyclictest26737-21munin-run21:15:141
1393299393,23cyclictest0-21swapper/121:05:261
13932993913,5cyclictest221ktimersoftd/123:55:141
1393299391,14cyclictest0-21swapper/119:15:271
1393499384,8cyclictest7950irq/56-dwc_otg_21:05:273
13934993829,6cyclictest29260-21apt-get00:20:253
13934993825,10cyclictest7126-21wget20:20:293
1393399381,24cyclictest16964-21ls20:45:392
1393399381,24cyclictest16964-21ls20:45:392
1393299389,12cyclictest1320-21chrt21:32:511
1393299386,22cyclictest0-21swapper/120:45:271
1393299386,22cyclictest0-21swapper/120:45:271
1393299386,20cyclictest0-21swapper/122:10:271
13932993818,4cyclictest19761-21wget20:55:281
13932993815,7cyclictest7850irq/56-dwc_otg_23:25:281
13934993727,8cyclictest8875-21apt-get20:25:253
13934993722,12cyclictest27190-21irqstats19:45:333
13934993712,15cyclictest0-21swapper/322:38:513
13933993711,8cyclictest0-21swapper/223:40:152
1393299379,12cyclictest1134-21chrt20:02:181
1393299373,22cyclictest7750irq/56-dwc_otg19:20:241
1393299373,22cyclictest7750irq/56-dwc_otg19:20:241
1393299373,13cyclictest28595-21sed21:20:141
1393299372,25cyclictest0-21swapper/121:57:591
13932993714,7cyclictest7750irq/56-dwc_otg00:35:341
13932993714,12cyclictest221ktimersoftd/100:01:371
1393299371,24cyclictest12682-21ls20:35:301
1393499367,9cyclictest7950irq/56-dwc_otg_19:35:243
13934993612,10cyclictest1125-21runrttasks20:47:183
13934993612,10cyclictest1125-21runrttasks20:47:183
13934993611,12cyclictest7850irq/56-dwc_otg_23:52:413
1393399367,19cyclictest0-21swapper/221:50:292
1393399364,4cyclictest7850irq/56-dwc_otg_22:30:142
1393399362,8cyclictest7750irq/56-dwc_otg23:25:252
1393399361,24cyclictest19280-21ls19:25:152
1393399361,24cyclictest19280-21ls19:25:152
1393299363,24cyclictest0-21swapper/122:46:271
13932993622,8cyclictest7850irq/56-dwc_otg_22:50:411
13932993622,8cyclictest7850irq/56-dwc_otg_22:50:411
1393299361,19cyclictest8282-21ls23:20:381
13932993611,4cyclictest0-21swapper/121:40:251
13932993610,6cyclictest12530-21chrt23:34:191
1393399358,6cyclictest7850irq/56-dwc_otg_23:40:342
1393399358,6cyclictest7850irq/56-dwc_otg_23:40:342
1393399355,5cyclictest7750irq/56-dwc_otg21:45:302
13933993513,6cyclictest12882-21if_err_wlan020:35:342
1393299357,19cyclictest7850irq/56-dwc_otg_00:25:131
1393299353,22cyclictest24681-21modprobe19:40:131
1393299353,16cyclictest7950irq/56-dwc_otg_21:35:271
1393299353,16cyclictest7950irq/56-dwc_otg_21:35:271
1393399347,14cyclictest0-21swapper/221:55:282
1393399341,24cyclictest5807-21missed_timers20:15:352
1393299348,5cyclictest0-21swapper/122:30:141
1393299342,19cyclictest0-21swapper/123:05:561
1393299341,20cyclictest0-21swapper/119:30:301
1393299341,16cyclictest9111-21fschecks_count20:25:311
1393299340,12cyclictest786-21taskset00:35:121
1393399338,5cyclictest7750irq/56-dwc_otg21:15:392
1393399337,14cyclictest7950irq/56-dwc_otg_20:40:372
1393399337,14cyclictest0-21swapper/219:30:272
1393399331,4cyclictest7850irq/56-dwc_otg_20:05:362
1393399331,25cyclictest18514-21missed_timers20:50:352
1393399331,22cyclictest30175-21timerandwakeup00:20:402
1393399331,22cyclictest27671-21cut00:15:332
1393399331,22cyclictest27671-21cut00:15:332
13932993312,6cyclictest0-21swapper/120:20:261
1393499326,18cyclictest0-21swapper/300:33:353
1393399325,7cyclictest7850irq/56-dwc_otg_22:25:142
1393399325,6cyclictest7850irq/56-dwc_otg_19:40:282
1393399323,17cyclictest0-21swapper/222:47:472
251732310,2sleep20-21swapper/222:40:142
1393399318,10cyclictest7950irq/56-dwc_otg_22:50:262
1393399318,10cyclictest7950irq/56-dwc_otg_22:50:262
1393399317,15cyclictest0-21swapper/219:25:282
1393399316,4cyclictest7850irq/56-dwc_otg_22:30:342
1393399313,20cyclictest0-21swapper/222:02:242
1393399311,5cyclictest7850irq/56-dwc_otg_22:55:342
1393399311,15cyclictest7850irq/56-dwc_otg_19:15:392
1393399309,5cyclictest7950irq/56-dwc_otg_23:20:302
1393399308,5cyclictest7950irq/56-dwc_otg_21:00:382
1393399306,17cyclictest7850irq/56-dwc_otg_23:00:392
1393399305,4cyclictest7950irq/56-dwc_otg_20:25:312
1393399302,19cyclictest0-21swapper/219:36:222
13933993015,4cyclictest17608-21/usr/sbin/munin23:45:412
1393399301,3cyclictest7750irq/56-dwc_otg00:10:362
1393399298,5cyclictest7750irq/56-dwc_otg23:15:352
1393399298,11cyclictest7950irq/56-dwc_otg_00:35:142
1393399296,4cyclictest7950irq/56-dwc_otg_21:05:282
1393399296,4cyclictest7850irq/56-dwc_otg_21:20:292
13933992923,4cyclictest1125-21runrttasks21:30:382
1393399291,4cyclictest29247-21/usr/sbin/munin19:50:352
123222297,11sleep30-21swapper/319:05:313
1393399286,4cyclictest25151-21/usr/sbin/munin21:10:382
1393399284,12cyclictest7950irq/56-dwc_otg_22:40:392
1393399281,9cyclictest0-21swapper/221:40:252
135002285,14sleep10-21swapper/119:08:161
1393399271,7cyclictest962-21grep20:00:422
13933992711,5cyclictest31352-21fschecks_time00:25:322
1393399266,5cyclictest7850irq/56-dwc_otg_23:30:352
1393399266,5cyclictest7850irq/56-dwc_otg_22:05:352
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional