You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-25 - 03:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Thu Dec 25, 2025 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1668522160,1sleep20-21swapper/219:08:552
250822830,2sleep00-21swapper/022:30:280
17051998212,53cyclictest762-21unixbench_singl00:20:340
1705199816,50cyclictest0-21swapper/020:35:210
17051998021,47cyclictest0-21swapper/023:25:230
17051997920,42cyclictest19404-21fschecks_count20:45:250
13812770,4sleep01375-21fschecks_count21:25:230
1705199767,45cyclictest5772-21grep23:05:330
1705199767,45cyclictest5772-21grep23:05:330
1705199749,62cyclictest1741-21wc19:55:330
1705199746,53cyclictest0-21swapper/023:55:350
1705199737,26cyclictest0-21swapper/000:25:120
1705199736,24cyclictest0-21swapper/023:30:220
17051997310,43cyclictest0-21swapper/019:45:220
1705199727,48cyclictest0-21swapper/022:05:240
1705199727,48cyclictest0-21swapper/022:05:240
1705199717,38cyclictest14007-21fschecks_count22:00:260
1705199717,38cyclictest14007-21fschecks_count22:00:260
1705199712,49cyclictest0-21swapper/000:10:370
1705299701,16cyclictest11730-21latency_hist21:55:081
1705199707,37cyclictest20033-21grep19:15:350
1705199707,37cyclictest20033-21grep19:15:350
17051997011,48cyclictest0-21swapper/022:35:190
1705599693,6cyclictest7850irq/56-dwc_otg_21:20:083
1705199698,42cyclictest0-21swapper/023:39:130
1705199697,49cyclictest0-21swapper/022:51:500
17051996922,26cyclictest0-21swapper/019:20:190
1705199691,47cyclictest4210-21sed23:00:360
1705299682,8cyclictest11876-21munin-run23:25:071
1705199689,38cyclictest3627-21ntpq21:30:300
17051996843,17cyclictest0-21swapper/000:35:190
1705399673,15cyclictest0-21swapper/222:25:192
1705599667,18cyclictest0-21swapper/323:25:283
1705199661,44cyclictest25964-21sed00:00:390
1705199661,39cyclictest0-21swapper/022:10:180
213312640,3sleep321325-21fschecks_time22:20:253
17055996413,40cyclictest7950irq/56-dwc_otg_21:55:083
1705199642,45cyclictest27236-21switchtime21:05:330
1705199639,41cyclictest0-21swapper/021:41:340
1705199639,40cyclictest20905-21basename23:50:080
1705199631,5cyclictest0-21swapper/023:40:200
1705199628,34cyclictest27661-21cron19:40:070
1705199627,45cyclictest0-21swapper/020:16:330
1705199622,32cyclictest11834-21latency_hist21:55:080
1705199621,25cyclictest0-21swapper/020:25:260
1705199616,41cyclictest25805-21taskset19:34:260
1705199615,42cyclictest0-21swapper/000:15:240
1705199612,37cyclictest22303-21grep23:50:360
1705199611,29cyclictest0-21swapper/020:05:210
17051996111,39cyclictest18137-21awk20:40:300
1705199609,40cyclictest24591-21latency_hist22:30:080
1705199602,29cyclictest0-21swapper/020:55:180
1705199601,29cyclictest0-21swapper/021:10:190
1705199601,29cyclictest0-21swapper/021:10:190
17051996010,36cyclictest32756-21head19:50:350
259252590,3sleep1221ktimersoftd/119:35:071
1705199598,30cyclictest20997-21/usr/sbin/munin22:20:250
1705199598,27cyclictest0-21swapper/022:45:280
1705199597,44cyclictest0-21swapper/021:17:150
1705199597,33cyclictest23586-21grep19:25:340
1705199591,42cyclictest0-21swapper/021:20:260
1705199591,41cyclictest0-21swapper/000:30:310
1705199591,38cyclictest6324-21sh21:40:070
1705599581,18cyclictest7950irq/56-dwc_otg_22:35:213
1705199587,28cyclictest18642-21sh19:15:080
1705199581,45cyclictest0-21swapper/020:11:200
1705199581,37cyclictest8607-21aten_rbpower_vo23:15:230
1705199579,36cyclictest0-21swapper/021:57:200
1705199579,25cyclictest0-21swapper/023:14:080
1705399567,28cyclictest0-21swapper/222:15:202
1705199561,40cyclictest10826-21cat23:20:270
1705199551,38cyclictest8890-21grep21:45:260
1705599547,28cyclictest0-21swapper/323:45:203
1705399547,24cyclictest0-21swapper/220:20:212
1705399541,2cyclictest0-21swapper/220:45:212
1705199549,22cyclictest0-21swapper/021:00:200
1705199548,32cyclictest22382-21chrt20:53:440
1705299535,9cyclictest7950irq/56-dwc_otg_19:15:201
1705299535,9cyclictest7950irq/56-dwc_otg_19:15:201
1705199537,9cyclictest0-21swapper/020:00:210
1705199533,39cyclictest0-21swapper/022:19:150
1705199526,35cyclictest28152-21chrt00:09:300
1705199526,35cyclictest28152-21chrt00:09:300
1705399515,10cyclictest0-21swapper/223:25:212
1705399513,4cyclictest0-21swapper/223:10:212
1705199513,39cyclictest0-21swapper/019:42:210
1705599507,24cyclictest0-21swapper/319:30:193
1705199507,32cyclictest2827-21taskset23:00:050
1705199502,34cyclictest0-21swapper/020:21:140
1705599497,28cyclictest0-21swapper/300:25:133
1705199486,12cyclictest15243-21chrt20:34:520
1705599477,22cyclictest0-21swapper/321:40:213
1705399477,25cyclictest0-21swapper/221:55:202
1705399477,25cyclictest0-21swapper/219:10:202
1705299477,27cyclictest0-21swapper/100:25:131
1705299477,26cyclictest0-21swapper/123:45:201
1705299471,33cyclictest7950irq/56-dwc_otg_20:30:231
1705199472,35cyclictest0-21swapper/022:43:320
1705399467,25cyclictest0-21swapper/223:40:212
1705399467,24cyclictest0-21swapper/219:50:132
1705299467,24cyclictest0-21swapper/120:25:141
1705399457,26cyclictest0-21swapper/222:50:222
1705399457,24cyclictest0-21swapper/200:00:142
1705399457,23cyclictest0-21swapper/222:20:212
1705399457,23cyclictest0-21swapper/220:15:212
1705399457,23cyclictest0-21swapper/219:25:192
1705399456,24cyclictest0-21swapper/223:20:202
1705399453,7cyclictest7750irq/56-dwc_otg21:45:182
1705299457,27cyclictest0-21swapper/122:05:201
1705299457,27cyclictest0-21swapper/122:05:201
17052994514,12cyclictest29865-21cron22:45:071
1705399447,24cyclictest0-21swapper/222:45:212
1705299447,26cyclictest0-21swapper/121:00:211
164392442,35sleep00-21swapper/019:05:470
1705599439,21cyclictest0-21swapper/320:45:083
1705399437,22cyclictest0-21swapper/222:30:132
1705399432,5cyclictest7950irq/56-dwc_otg_22:05:192
1705399432,5cyclictest7950irq/56-dwc_otg_22:05:192
1705299431,2cyclictest0-21swapper/122:35:211
17055994219,5cyclictest17850-21irqstats22:10:263
1705399423,6cyclictest0-21swapper/220:30:212
1705299422,4cyclictest7950irq/56-dwc_otg_23:55:201
17052994224,5cyclictest21065-21grep20:50:241
262162410,2sleep30-21swapper/319:35:223
1705599417,22cyclictest0-21swapper/320:50:233
1705399417,22cyclictest0-21swapper/200:25:212
1705299417,23cyclictest0-21swapper/121:40:211
1705299413,11cyclictest0-21swapper/120:08:351
183072400,5sleep11705299cyclictest22:10:331
17055994020,13cyclictest9498-21seq20:17:013
1705399407,23cyclictest0-21swapper/221:25:202
1705399407,23cyclictest0-21swapper/221:05:212
1705399401,5cyclictest7950irq/56-dwc_otg_20:05:242
17052994032,6cyclictest221ktimersoftd/121:30:201
17053993916,6cyclictest7157-21uname20:10:332
1705299397,26cyclictest0-21swapper/100:20:201
1705299394,4cyclictest10073-21munin-run23:20:071
17052993930,6cyclictest221ktimersoftd/100:35:181
17052993920,10cyclictest7850irq/56-dwc_otg_19:26:121
17052993914,6cyclictest21-21rcuc/123:40:361
1705599381,4cyclictest7750irq/56-dwc_otg19:15:183
1705599381,4cyclictest7750irq/56-dwc_otg19:15:183
1705399387,6cyclictest7850irq/56-dwc_otg_00:35:202
1705399383,5cyclictest7750irq/56-dwc_otg00:15:202
17053993819,5cyclictest4749-21/usr/sbin/munin21:35:272
17053993811,6cyclictest7750irq/56-dwc_otg19:45:252
1705299386,22cyclictest0-21swapper/121:20:201
1705299384,16cyclictest13670-21ls22:00:081
1705599372,23cyclictest0-21swapper/321:20:203
1705599371,21cyclictest28767-21missed_timers21:10:293
1705599371,21cyclictest28767-21missed_timers21:10:293
17053993716,5cyclictest28311-21/usr/sbin/munin22:40:212
1705299378,5cyclictest7850irq/56-dwc_otg_00:35:081
1705299376,13cyclictest7950irq/56-dwc_otg_22:30:371
17052993714,5cyclictest15618-21aten_rbpower_vo20:35:221
17052993714,4cyclictest17149-21aten_rbpower_po19:10:211
1705599367,21cyclictest0-21swapper/300:20:203
1705399369,16cyclictest0-21swapper/223:00:212
1705299369,7cyclictest21-21rcuc/120:20:251
1705299368,5cyclictest7850irq/56-dwc_otg_21:20:071
15682360,2sleep20-21swapper/222:55:262
84912350,5sleep31705599cyclictest23:15:193
1705599356,6cyclictest7850irq/56-dwc_otg_22:50:223
1705599353,25cyclictest7950irq/56-dwc_otg_20:55:183
1705599352,4cyclictest7850irq/56-dwc_otg_22:55:203
1705599351,25cyclictest3456-21ls21:30:283
1705399356,7cyclictest7750irq/56-dwc_otg19:15:282
1705399356,7cyclictest7750irq/56-dwc_otg19:15:282
1705399356,5cyclictest29-21rcuc/223:30:232
1705399356,19cyclictest7850irq/56-dwc_otg_22:10:182
1705299353,8cyclictest17444-21aten_rbpower_vo20:40:201
17052993515,10cyclictest25762-21sed00:00:371
1705599347,24cyclictest0-21swapper/323:55:213
1705599346,14cyclictest0-21swapper/321:00:213
1705599344,21cyclictest29550-21grep00:10:373
17055993415,11cyclictest28320-21if_eth019:40:273
17055993411,9cyclictest22122-21timerwakeupswit23:50:353
1705399348,5cyclictest7850irq/56-dwc_otg_23:55:292
1705399345,7cyclictest7850irq/56-dwc_otg_20:40:342
1705299348,6cyclictest31969-21/usr/sbin/munin19:50:251
1705299347,23cyclictest0-21swapper/121:45:201
17052993412,5cyclictest16318-21cat23:35:291
1705299341,21cyclictest2845-21latency_hist23:00:081
1705599337,14cyclictest0-21swapper/322:05:203
1705599337,14cyclictest0-21swapper/322:05:203
1705599333,7cyclictest7950irq/56-dwc_otg_20:25:273
17055993313,13cyclictest169072chrt20:38:013
1705399336,15cyclictest0-21swapper/221:15:212
1705399333,8cyclictest0-21swapper/221:10:192
1705399333,8cyclictest0-21swapper/221:10:192
1705299336,10cyclictest7850irq/56-dwc_otg_23:00:361
1705299335,11cyclictest7950irq/56-dwc_otg_22:15:241
17052993313,4cyclictest9768-21latency_hist20:20:081
17052993312,13cyclictest221ktimersoftd/119:55:291
1705299331,19cyclictest29737-21latency_hist21:15:081
1705299331,19cyclictest29737-21latency_hist21:15:081
17055993216,5cyclictest30390-21fschecks_count22:45:243
17055993213,11cyclictest7959-21latency_hist20:15:083
1705599321,22cyclictest26412-21latency_hist00:05:083
17055993210,15cyclictest21423-21open_inodes19:20:313
1705399326,15cyclictest7950irq/56-dwc_otg_00:20:262
1705399326,15cyclictest0-21swapper/219:20:212
1705399323,5cyclictest7850irq/56-dwc_otg_19:40:072
17053993212,4cyclictest7750irq/56-dwc_otg23:05:242
17053993212,4cyclictest7750irq/56-dwc_otg23:05:242
1705399321,20cyclictest3902-21timerandwakeup21:30:352
1705299322,4cyclictest28218-21tune2fs19:40:251
1705299322,24cyclictest15359-21ls22:05:081
1705299322,24cyclictest15359-21ls22:05:081
1705299321,5cyclictest0-21swapper/121:05:351
1705299321,22cyclictest31842-21latency_hist00:20:081
70232310,3sleep11705299cyclictest20:10:291
1705599317,21cyclictest0-21swapper/321:45:203
1705599316,5cyclictest28627-21fschecks_time22:40:253
1705599314,6cyclictest7950irq/56-dwc_otg_00:35:173
1705599312,7cyclictest7950irq/56-dwc_otg_23:15:073
17055993115,4cyclictest17911-21/usr/sbin/munin19:10:323
17055993114,11cyclictest381ktimersoftd/323:31:023
1705599311,16cyclictest7950irq/56-dwc_otg_23:40:513
1705399317,14cyclictest0-21swapper/220:25:222
1705399316,14cyclictest7950irq/56-dwc_otg_00:30:222
1705399314,6cyclictest7850irq/56-dwc_otg_20:00:072
1705399311,22cyclictest3032-21cut20:00:272
1705299313,15cyclictest31991-21aten_rbpower_po22:50:221
17052993125,4cyclictest15474-21sh23:35:071
1705299312,4cyclictest22184-21users22:20:361
1705299311,7cyclictest3056-21fschecks_count20:00:271
17052993112,4cyclictest7094-21if_eth023:10:261
17052993111,5cyclictest5570-21ntpq23:05:301
17052993111,5cyclictest5570-21ntpq23:05:301
301302300,3sleep10-21swapper/122:45:181
1705599303,6cyclictest7950irq/56-dwc_otg_19:50:283
1705599303,5cyclictest7850irq/56-dwc_otg_20:25:073
1705599302,7cyclictest7950irq/56-dwc_otg_19:25:203
1705599301,5cyclictest7850irq/56-dwc_otg_20:35:083
1705399307,14cyclictest0-21swapper/220:55:212
1705399307,14cyclictest0-21swapper/219:40:222
1705399307,14cyclictest0-21swapper/200:05:212
1705399307,14cyclictest0-21swapper/200:05:212
1705399305,12cyclictest7950irq/56-dwc_otg_21:20:202
17053993014,5cyclictest19599-21fschecks_count23:45:242
1705399301,19cyclictest16401-21switchtime20:35:332
17053993010,6cyclictest7850irq/56-dwc_otg_19:30:262
1705299305,5cyclictest23-21ksoftirqd/119:45:261
1705299301,6cyclictest27345-21ntp_states00:05:311
1705299301,6cyclictest27345-21ntp_states00:05:311
1705299301,5cyclictest26569-21irqstats19:35:261
1705299301,4cyclictest4749-21/usr/sbin/munin21:35:321
17052993012,10cyclictest1136-21sshd23:27:421
1705599299,5cyclictest5553-21ntp_kernel_pll_23:05:303
1705599299,5cyclictest5553-21ntp_kernel_pll_23:05:303
1705599294,4cyclictest7750irq/56-dwc_otg22:30:353
1705599292,8cyclictest0-21swapper/320:48:463
17055992921,6cyclictest1125-21runrttasks22:00:233
17055992921,6cyclictest1125-21runrttasks22:00:233
1705599291,7cyclictest4822-21aten_rbpower_en21:35:223
1705599291,4cyclictest11885-21idleruntime-cro23:25:073
1705599291,26cyclictest24448-21ls22:30:083
17055992911,11cyclictest1125-21runrttasks19:57:363
17055992910,10cyclictest381ktimersoftd/323:35:313
1705399298,9cyclictest7750irq/56-dwc_otg21:40:292
1705399292,19cyclictest0-21swapper/222:36:162
1705399292,19cyclictest0-21swapper/222:03:272
1705399292,19cyclictest0-21swapper/222:03:272
1705399291,20cyclictest10487-21cut21:50:252
1705299295,4cyclictest7950irq/56-dwc_otg_00:10:201
1705299294,9cyclictest19484-21tune2fs20:45:251
1705299293,6cyclictest1469-21fschecks_time21:25:241
1705299293,4cyclictest0-21swapper/122:25:331
1705299292,4cyclictest21128-21/usr/sbin/munin23:50:321
17052992923,4cyclictest21022-21fschecks_time19:20:251
1705299291,15cyclictest0-21swapper/120:57:181
1705599288,7cyclictest0-21swapper/300:05:363
1705599288,7cyclictest0-21swapper/300:05:363
1705599286,4cyclictest7850irq/56-dwc_otg_21:55:263
1705599286,12cyclictest7950irq/56-dwc_otg_20:05:303
1705599282,4cyclictest0-21swapper/321:25:243
17055992822,4cyclictest30373-21aten_rbpower_vo00:15:213
1705599281,5cyclictest3075-21grep20:00:273
1705599281,5cyclictest1124-21snmpd22:15:283
1705399287,11cyclictest7950irq/56-dwc_otg_20:50:232
1705399282,17cyclictest0-21swapper/223:51:552
1705399281,5cyclictest7850irq/56-dwc_otg_23:35:252
1705599271,5cyclictest4919-21modprobe00:35:073
1705399274,5cyclictest7750irq/56-dwc_otg00:10:302
1705399271,5cyclictest0-21swapper/221:02:022
164552273,15sleep10-21swapper/119:06:001
151162272,14sleep30-21swapper/319:05:093
1705599265,19cyclictest26712-21idleruntime21:05:263
1705599261,5cyclictest7950irq/56-dwc_otg_23:00:223
1705399266,4cyclictest7750irq/56-dwc_otg23:15:252
17055992312,4cyclictest29693-21modprobe19:45:083
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional