You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 21:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TI
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6s.osadl.org (updated Sat Jun 28, 2025 12:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2938124998047,28cyclictest2957230-21sendmail09:00:010
2938124998028,23cyclictest0-21swapper/010:45:000
2938124997322,41cyclictest0-21swapper/010:00:330
2938124997139,22cyclictest2939035-21wc07:15:020
2938124996728,34cyclictest0-21swapper/009:54:590
2938124996724,33cyclictest0-21swapper/010:10:300
2938124996718,39cyclictest0-21swapper/012:20:310
2938124996529,30cyclictest0-21swapper/010:45:300
2938124996431,29cyclictest20950irq/209-8000000.ethernet11:20:290
2938124996418,40cyclictest0-21swapper/012:10:320
2938124996418,34cyclictest0-21swapper/007:40:300
293812499628,13cyclictest2956315-21grep08:50:310
2938124996230,29cyclictest20950irq/209-8000000.ethernet11:50:280
2938124996229,29cyclictest20950irq/209-8000000.ethernet12:05:270
2938124996227,28cyclictest0-21swapper/011:30:300
2938124996224,30cyclictest0-21swapper/010:25:340
293812499621,53cyclictest2988000-21grep11:55:300
293812499614,7cyclictest2964856-21unixbench_singl09:40:330
293812499614,7cyclictest2964856-21unixbench_singl09:40:330
293812499612,51cyclictest2949645-21latency_hist08:15:010
2938124996122,3cyclictest25250irq/422-mmc112:35:060
293812499602,49cyclictest2974586-21apt-config10:40:020
2938124996023,32cyclictest0-21swapper/009:35:320
293812499602,18cyclictest2965722-21grep09:45:320
2938124995912,7cyclictest0-21swapper/012:00:320
293812499591,10cyclictest2970402-21/usr/sbin/munin10:15:240
2938124995821,32cyclictest0-21swapper/009:00:330
293812499575,11cyclictest2991441-21grep12:15:320
2938124995730,23cyclictest20950irq/209-8000000.ethernet08:35:240
2938124995720,26cyclictest0-21swapper/011:35:310
293812499571,8cyclictest0-21swapper/008:45:020
2938124995714,8cyclictest0-21swapper/011:40:290
293812499562,11cyclictest2955430-21grep08:45:320
293812499561,51cyclictest2961962-21/usr/sbin/munin09:25:250
293812499561,51cyclictest2961962-21/usr/sbin/munin09:25:240
2938128995516,5cyclictest2975336-21apt-get10:45:022
293812499554,10cyclictest2945186-21head07:45:330
293812499554,10cyclictest2945186-21head07:45:320
2938124995517,29cyclictest0-21swapper/009:05:320
2938124995419,4cyclictest2962586-21/usr/sbin/munin09:30:170
2938124995414,9cyclictest0-21swapper/007:30:300
293812499537,15cyclictest2948596-21apt-get08:10:010
293812499532,8cyclictest2977859-21grep10:55:310
293812499532,8cyclictest2977859-21grep10:55:310
2938124995328,20cyclictest2951175-21grep08:20:320
293812499532,4cyclictest2939482-21df_inode07:15:200
293812499532,4cyclictest2939482-21df_inode07:15:200
293812499531,45cyclictest0-21swapper/008:00:250
293812499531,45cyclictest0-21swapper/008:00:250
2938124995314,5cyclictest0-21swapper/007:20:260
2938128995241,8cyclictest2986408-21munin-plugin-st11:50:002
293812499527,11cyclictest0-21swapper/007:55:210
293812499523,13cyclictest2952858-21grep08:30:300
2938124995220,5cyclictest2976492-21cpu10:50:210
293812499503,9cyclictest2981223-21tail11:15:350
293812499501,22cyclictest2973163-21irqstats10:30:240
293812499501,21cyclictest0-21swapper/009:20:150
2938124995011,6cyclictest0-21swapper/010:05:290
2938128994940,4cyclictest2950254-21unixbench_multi08:15:332
2938128994919,18cyclictest311ktimers/207:25:142
293812499492,4cyclictest2945479-21df07:50:230
293812499492,12cyclictest2971748-21tail10:20:310
2938124994917,7cyclictest293-21systemd-journal12:30:000
2938124994917,7cyclictest293-21systemd-journal12:30:000
293812499491,46cyclictest2951906-21grep08:25:300
293812899485,5cyclictest0-21swapper/207:55:192
293812699481,8cyclictest0-21swapper/109:45:001
293812699481,8cyclictest0-21swapper/109:45:001
2938124994818,6cyclictest2943506-21sshd07:37:390
2938128994735,8cyclictest0-21swapper/208:00:312
2938128994735,8cyclictest0-21swapper/208:00:312
2938124994718,4cyclictest2950557-21grep08:20:010
293812499471,22cyclictest0-21swapper/009:55:130
2938126994634,8cyclictest2950384-21sendmail08:20:001
2938126994634,8cyclictest0-21swapper/109:30:341
2938126994632,9cyclictest0-21swapper/109:15:301
293812699463,25cyclictest24-21ksoftirqd/108:35:021
293812499462,12cyclictest2979642-21cron11:10:000
2938124994620,3cyclictest0-21swapper/009:13:580
2938124994617,4cyclictest0-21swapper/011:45:310
2938124994616,6cyclictest2978717-21grep11:00:350
293812499461,24cyclictest0-21swapper/011:25:220
293812499461,22cyclictest0-21swapper/007:25:400
2938128994538,5cyclictest311ktimers/208:55:202
2938126994531,9cyclictest0-21swapper/110:45:311
293812699453,11cyclictest22-21rcuc/111:25:001
293812499452,10cyclictest2960715-21grep09:19:590
293812499451,38cyclictest0-21swapper/011:10:180
2938128994432,8cyclictest0-21swapper/210:50:342
2938128994430,8cyclictest0-21swapper/210:05:312
2938126994431,8cyclictest0-21swapper/107:40:311
2938126994426,13cyclictest231ktimers/111:15:171
2938128994331,8cyclictest0-21swapper/207:30:312
2938128994330,9cyclictest0-21swapper/209:45:322
2938126994336,4cyclictest2994090-21munin-plugin-st12:35:001
2938126994336,3cyclictest2946907-21sendmail08:00:001
2938126994331,8cyclictest0-21swapper/108:40:311
293812499431,22cyclictest0-21swapper/012:30:290
2938128994234,5cyclictest2989783-21munin-plugin-st12:10:002
2938128994232,4cyclictest0-21swapper/208:40:212
2938128994231,7cyclictest0-21swapper/212:20:322
2938128994230,9cyclictest0-21swapper/211:40:292
2938128994229,9cyclictest0-21swapper/207:15:302
2938128994229,9cyclictest0-21swapper/207:15:302
2938128994225,8cyclictest0-21swapper/210:25:342
2938126994234,5cyclictest0-21swapper/108:45:291
2938126994230,8cyclictest0-21swapper/107:39:591
2938126994228,10cyclictest0-21swapper/107:25:001
2938128994133,5cyclictest2970931-21sendmail10:19:592
2938128994128,9cyclictest0-21swapper/210:35:332
2938128994128,8cyclictest311ktimers/209:30:162
2938128994123,11cyclictest0-21swapper/212:10:322
2938126994129,7cyclictest0-21swapper/112:10:011
2938128994025,7cyclictest0-21swapper/211:50:332
2938128993935,2cyclictest0-21swapper/208:35:022
2938128993929,7cyclictest0-21swapper/209:15:312
2938128993926,9cyclictest311ktimers/209:35:172
2938128993926,9cyclictest0-21swapper/211:25:332
2938126993929,5cyclictest0-21swapper/111:05:001
2938128993828,6cyclictest0-21swapper/211:20:332
2938128993826,8cyclictest0-21swapper/210:55:312
2938128993826,8cyclictest0-21swapper/210:55:312
2938128993826,8cyclictest0-21swapper/208:50:302
2938128993825,9cyclictest0-21swapper/209:05:312
2938128993825,8cyclictest0-21swapper/208:20:332
2938128993825,8cyclictest0-21swapper/208:10:302
2938128993821,8cyclictest0-21swapper/210:10:302
2938126993830,5cyclictest0-21swapper/112:15:001
2938126993830,5cyclictest0-21swapper/110:30:171
2938126993828,5cyclictest0-21swapper/108:25:001
2938126993827,5cyclictest0-21swapper/110:00:341
2938126993824,9cyclictest0-21swapper/108:35:271
2938126993819,15cyclictest231ktimers/110:05:141
2938128993729,5cyclictest168450irq/135-8000000.ethernet-tx211:20:002
2938128993725,7cyclictest0-21swapper/212:25:342
2938128993725,7cyclictest0-21swapper/212:25:342
2938128993724,9cyclictest0-21swapper/209:00:332
2938128993724,9cyclictest0-21swapper/208:10:002
2938126993728,5cyclictest2992328-21grep12:20:321
2938126993728,4cyclictest0-21swapper/111:35:211
2938126993726,7cyclictest0-21swapper/110:40:001
2938126993724,8cyclictest0-21swapper/107:10:341
2938126993723,7cyclictest0-21swapper/107:45:321
2938126993723,7cyclictest0-21swapper/107:45:321
2938126993720,9cyclictest0-21swapper/109:35:311
2938128993628,4cyclictest0-21swapper/207:40:002
2938128993626,7cyclictest0-21swapper/210:05:012
2938128993623,8cyclictest0-21swapper/209:30:022
2938128993623,8cyclictest0-21swapper/209:30:012
2938128993613,3cyclictest27350irq/343-s-wl18xx12:35:052
2938126993628,4cyclictest0-21swapper/111:45:201
2938126993627,4cyclictest0-21swapper/111:05:351
2938126993621,8cyclictest2957146-21unixbench_singl08:55:291
293812699361,4cyclictest0-21swapper/109:25:011
2938128993526,5cyclictest14-21rcu_preempt10:20:212
2938128993522,9cyclictest0-21swapper/210:45:302
2938128993522,8cyclictest0-21swapper/211:55:212
2938126993525,7cyclictest2994972-21grep12:35:311
2938126993525,7cyclictest0-21swapper/110:45:011
2938126993524,7cyclictest0-21swapper/109:05:321
2938126993523,8cyclictest0-21swapper/112:25:341
2938126993523,8cyclictest0-21swapper/112:25:341
2938126993519,11cyclictest231ktimers/111:25:201
293812499351,2cyclictest0-21swapper/007:10:010
293812899349,10cyclictest14-21rcu_preempt07:50:022
293812899349,10cyclictest14-21rcu_preempt07:50:022
2938128993426,6cyclictest2959797-21unixbench_singl09:10:312
2938128993421,7cyclictest0-21swapper/211:10:242
2938126993426,5cyclictest0-21swapper/108:25:321
2938126993426,4cyclictest0-21swapper/109:00:281
2938126993422,6cyclictest2972623-21grep10:25:331
2938126993420,9cyclictest231ktimers/112:00:151
2938126993420,5cyclictest0-21swapper/108:10:001
2938128993327,3cyclictest168450irq/135-8000000.ethernet-tx209:20:122
2938128993324,7cyclictest2938290-21/usr/sbin/munin07:10:292
2938128993321,9cyclictest0-21swapper/211:35:312
2938128993318,8cyclictest0-21swapper/212:30:202
2938128993318,8cyclictest0-21swapper/207:40:302
2938128993313,4cyclictest27350irq/343-s-wl18xx11:00:222
2938126993326,3cyclictest0-21swapper/107:30:221
2938126993324,4cyclictest0-21swapper/110:10:291
2938126993324,4cyclictest0-21swapper/109:45:191
2938126993322,5cyclictest0-21swapper/111:40:201
2938126993320,8cyclictest0-21swapper/112:00:011
2938128993222,7cyclictest2946025-21grep07:50:342
2938128993212,4cyclictest27350irq/343-s-wl18xx09:43:092
2938128993212,4cyclictest27350irq/343-s-wl18xx09:43:082
2938126993224,4cyclictest0-21swapper/109:55:311
2938126993223,7cyclictest0-21swapper/110:20:311
2938128993122,6cyclictest0-21swapper/208:25:322
2938128993114,13cyclictest0-21swapper/211:30:312
293812899311,3cyclictest2966477-21find09:50:322
293812899311,28cyclictest0-21swapper/211:05:342
2938128993111,4cyclictest27350irq/343-s-wl18xx09:56:032
2938126993126,4cyclictest2938118-21cyclictest12:15:311
2938126993123,5cyclictest0-21swapper/111:10:331
2938126993121,7cyclictest0-21swapper/110:15:301
293812699311,27cyclictest2976466-21/usr/sbin/munin10:50:291
2938128993018,9cyclictest0-21swapper/210:30:302
293812899301,27cyclictest0-21swapper/212:15:302
2938126993021,6cyclictest0-21swapper/110:55:231
2938126993021,6cyclictest0-21swapper/110:55:231
2938126993015,8cyclictest0-21swapper/109:55:001
293812699301,27cyclictest0-21swapper/111:30:241
2938128992919,8cyclictest311ktimers/207:20:222
2938126992921,4cyclictest0-21swapper/111:50:341
2938126992919,7cyclictest0-21swapper/107:50:271
293812699291,14cyclictest0-21swapper/109:25:501
293812699291,14cyclictest0-21swapper/109:25:501
2938128992822,3cyclictest2988421-21fschecks_count12:00:222
2938128992819,5cyclictest0-21swapper/208:45:312
2938128992720,4cyclictest0-21swapper/208:35:342
2938126992721,4cyclictest2938917-21kworker/u8:1-events_freezable_power_07:15:281
2938126992721,4cyclictest2938917-21kworker/u8:1-events_freezable_power_07:15:281
2938126992720,4cyclictest22-21rcuc/108:10:291
2938126992717,6cyclictest231ktimers/108:00:311
2938126992717,6cyclictest231ktimers/108:00:301
293812699251,15cyclictest0-21swapper/109:13:141
2938126992418,4cyclictest22-21rcuc/108:50:251
2938126992417,5cyclictest0-21swapper/107:30:001
2938126992417,4cyclictest0-21swapper/107:10:011
2938128992315,6cyclictest0-21swapper/207:10:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional