Archive for the ‘Uncategorized’ Category

Apple Upset – upgrading to iPhone 7

December 22nd, 2016

Upgrading is always stressful – be it a computer, an Oracle database or an iPhone. There’s always a good chance for lost data and lost time dealing with complications.

So yesterday I picked up a new iPhone 7 from Verizon. The pickup was seamless. I had signed up for an upgrade program when I got the iPhone 6, so now I just walked in, gave them my old iPhone 6 and they gave me an new iPhone 7. It’s bit scary giving up my old phone before restoring to my new phone, but I had a backup AND I asked Verizon to please not wipe my iPhone 6 for 24  hours incase there were upgrade errors. They normally wipe the phone immediately.

The day was off to a good start. It only took about 10 minutes to get the phone and I had taken a full backup of my iPhone 6 the day before and thought I’d plug in , restore back up and wow, that would be easy.

I get back to my office, luckily just a couple blocks away. Plug it in, try to restore the backup and it asked me for a password. I’m like ‘rrr’! The day before when I had taken the backup, I saw that the “encrypt” checkbox was filled and thought about taking it off, but then thought, “well it’s probably more prudent to leave it on”. Of course the backup didn’t ask me to verify my password. It just took the backup.


Now in order to use the backup, I had to know the password. The day before when it took the backup, I was thinking my computer had it cached, or else why didn’t it ask for the password when I took the backup?

So now I had to figure out what the password it was. I tried all conceivable passwords I could think of. My name space of passwords is limited to about 3 core, 10 common  and 20 rare passwords. I tried them all. It wasn’t my Apple ID password which it should have been. It wasn’t my iPhone 6 number code. It wasn’t any of the 20 passwords I’ve used over the past several years.

OK, fine. I’ll go back get my iPhone 6 and take a new un-encrypted backup!

I go back to the store. To their credit they gave me my old phone in a couple of minutes and it was back to the office.

I go to the backup interface, seen above, and try to un-check the “Encrypt iPhone backup”.

This is  where it gets fun.

*** No ***

you can’t uncheck the Encrypt!  What has been seen can never be unseen.

To uncheck the box you need the password. If I knew the password I wouldn’t be unchecking the box.

So now I  *** CAN’T *** take a backup (at least one that is usable)! Thanks a lot Apple. Are you serious?

It’s my  iPhone, my computer, I’m using everything on both and Apple won’t let me take a backup of the iPhone on the computer!

From Apple:


Unbelievable. There is no way for me to backup my iPhone 6 so I can upgrade to my iPhone 7.

Are they serious ??

The work around is to backup to the iCloud which doesn’t need the password. Does that make sense? Can we say slow, insecure and inefficient?

Backing up my 64G  iPhone 6 to the iCloud is a recipe for a huge waste of time.

So I go through and blow everything away on my iPhone (so much for a good backup) until I’m down to about 6G and then backup to the iCloud. Of course, I have to buy more space on the iCloud to do this. rrrr

I bought more space, I backed up the iPhone, I went back to Verizon, switched phones again, back to the office, restored from iCloud and it worked. Of course I’m missing all the photos, music, books and apps I rarely used. Now time to put that stuff on back by hand.

Some parts of the upgrade are magic, but this part blows me away. Why? Why so much pain? Just to force me to buy some iCloud space for a day?

What a frustrating waste of time from a company that prides itself in easy powerful user interfaces.

First, why can’t you turn of encryption of backups once chosen?

Second why not use apple ID password ?

Third, why not alert the user every time they take an encrypted backup asking for password? No point in taking the encrypted backup if you don’t know what the password is.



Why does my full table scan take 10x longer today ?!

December 13th, 2016

Every so often a DSS query that usually takes 10 minutes ends up taking over an hour.  (or one that takes an hour never seems to finish)

Why would this happen?

When investigating the DSS query, perhaps with wait event tracing,  one finds that the query which is doing full table scans and should be doing large multi-block reads and waiting for “db file scattered read” is instead waiting for single block reads, ie “db file sequential read”.  What the heck is going on?

Sequential reads during a  full table scan scattered read query is a classic sign of reading rollback and reading rollback can make that minute(s) full table scan take hours.

What can happen especially after over night jobs, is that if an overnight job fails to finished before the DSS query is run and if that overnight job  does massive updates without committing till then end, then the DSS query will have to rollback any changes made by the updates to the tables the DSS query is accessing.

How do we quickly identify if this our issue?

ASH is good at identify it. On the other hand it’s often impractical to whip up from scratch an ASH query and that’s where ashmasters on Github comes in. This ASH query and others are on Github under ashmasters.


For this case specifically see:

Here is the output (slight different format than in the github repository) of a query I used in my Oracle Performance classes

AAS SQL_ID           %  OBJ              TABLESPACE
----- -------------  ---  ---------------  ----------
  .18 0yas01u2p9ch4    6  ITEM_PRODUCT_IX  SOEINDEX
                       6  ORDER_ITEMS_UK   SOEINDEX
                      88  ITEM_ORDER_IX    SOEINDEX
  .32 6v6gm0fd1rgrz    6  MY_BIG_Table     SOEDATA
                      94  UNDO             UNDOTBS1

i.e. 95% of the second SQL_ID’s i/o was coming from UNDO. The reads will be single block reads and tremendously slow down the full table scans.


Amazon announces Performance Insights

December 2nd, 2016

Excited to see the announcement of Amazon RDS Performance Insight feature for database performance monitoring and tuning.

Having met the team for this project I can say from my personal view point that the importance and future success of this feature is clear as day to me. The team is awesomely sharp, the architecture is super impressive, and this is by far the most exciting performance monitoring and feedback system I’ve been involved with,  surpassing the work I’ve been involved in on Oracle’s performance monitoring and tuning system and Embarcadero’s DB Optimizer and Quest Foglight and Spotlight. Not only does the feature provide it’s own dashboard but will also provide an API to power your dashboards that already exist in the industry. I expect to see partners leveraging the API to provide new insights in their already existing database performance monitors.

Below are a couple of snippets that showed up on the web this week  as well as a couple of photos of Jeremiah Wilton, who I personally consider the catalyst and moving force behind this feature, giving demonstrations of the feature.

PS the team is also hiring! If you you are interested, check out this post on ycombinator.

On Amazon Blog


On Brent Ozar’s  blog

 Amazon’s throwing in free performance monitoring tools. The new Performance Insights tool shows wait stats, top resource-intensive queries, lock detection, execution plans, and 35 days of data retention. It’s designed by a former Oracle (there we go again) performance tuning consultant to match his workflow. You don’t have to install an agent, configure a repository, or keep the thing running. It’s in preview for Aurora PostgreSQL today, and will be rolled out to all RDS databases (including SQL Server) in 2017.




Live Demo at the re:Invent demo grounds


Video of presentation at re:Invent


Starting at 39:00



SQL*PLus on Mac

December 1st, 2016

I would think installing SQL*Plus on the Mac would be point, click download, point click, install bam it works.


It did install mostly straight forward on my old Mac. Got a new Mac and no dice.

Tried installing myself guessing at the downloads. First of all why isn’t there just one download?

Downloaded instantclient and instantclient with SQL*Plus which turns out to be correct, but no dice. Still got errors.

Got errors. Gave up.

Came back again to look at it yesterday and followed this:

worked like a charm.

Then I ran a shell script that used SQL*Plus and get the error

dyld: Library not loaded: /ade/dosulliv_sqlplus_mac/oracle/sqlplus/lib/libsqlplus.dylib
Referenced from: /Applications/oracle/product/instantclient_64/
Reason: image not found

so I set my environment variable


Let’s check the environment variable:

$ env | grep LIB

Huh?! Where is my environment variable?

Apparently you can’t set DYLD_LIBRARY_PATH on Mac without over riding some security settings which didn’t sound attractive.

I googled around and found

which didn’t work for me. Then found

which has as a solution to set the library path inside an alias for SQL*Plus – cool!

alias sqlplus=”DYLD_LIBRARY_PATH=/Applications/oracle/product/instantclient_64/ sqlplus”

and I put that into my ~/.bashrc and it worked!


Started at Amazon! … want to join me?

August 29th, 2016

(Disclamer: any opinions expressed here are fully my own and not representative of my employer)


photo by alvaroprieto  (cc 2.0)

Super excited to be working at Amazon on my passion which is performance data visualization and database monitoring. Suffice it to say this is the most excited I’ve been about work in my career and I’ve had ample opportunity to work on database performance in the past such as at Oracle (where I helped design the performance pages and designed Top Activity page), at Quest (now Dell) on Spotlight, on my own free tools ( ASHMon, S-ASH, W-ASH, Oramon etc)  and at Embarcadero where our team produced DB Optimizer that extended sampling and average active sessions to SQL Server, DB2 and Sybase (not to mention Visual SQL Tuning). The work here at Amazon looks to largely surpass all previous work.

More to news to come as I settle in.

In the mean time Amazon is looking to hire! We are looking for product managers, developers, service people etc. Mainly senior people with a good track record.  Please feel free to contact me if (and only if)

  • you are senior in your career  and/or
  • we have personally worked together  and/or
  • you have done something innovative already in your career (a free tool, a new design, etc).

Please refrain from contacting me about junior positions.  If you are interested in junior positions please look at the Amazon jobs listed on their website. Amazon is hiring aggressively!

These positions are almost all out of Seattle. There is some chance of working in Vancouver and Palo Alto though it would be recommended to work out of Seattle.

One specific position on my groups team is a Data Engineer to work on reporting. Here is the job listing from the Amazon site:


External job description:

Amazon Relational Database Service (Amazon RDS) is an industry leading web service that makes it easy to set up, operate, and scale a relational database in the cloud using any of the leading database engines – MySQL, MariaDB, PostgreSQL, SQL Server and Oracle, as well as Amazon’s own MySQL-compatible database engine, Aurora. We are looking for a for a seasoned and talented data engineer to join the team in our Seattle Headquarters. More information on Amazon RDS is available at

The data engineer must be passionate about data and the insights that large amounts of data can provide and has the ability to contribute major novel innovations for our team. The role will focus on working with a team of product and program managers, engineering leaders and business leaders to build pipelines and data analysis tools to help the organization run it’s business better. The role will focus on business insights, deep data and trend analysis, operational monitoring and metrics as well as new ideas we haven’t had yet (but you’ll help us have!). The ideal candidate will possess both a data engineering background and a strong business acumen that enables him/her to think strategically and add value to help us improve the RDS customer experience. He/she will experience a wide range of problem solving situations, strategic to real-time, requiring extensive use of data collection and analysis techniques such as data mining and machine learning. In addition, the data engineering role will act as a foundation for the business intelligence team and be forward facing to all levels within the organization.

· Develop and improve the current data architecture for RDS
· Drive insights into how our customers use RDS, how successful they are, where our revenue trends are going up or down, how we are helping customers have a remarkable experience, etc.
· Improve upon the data ingestion models, ETLs, and alarming to maintain data integrity and data availability.
· Keep up to date with advances in big data technologies and run pilots to design the data architecture to scale with the increased data sets of RDS.
· Partner with BAs across teams such as product management, operations, sales, marketing and engineering to build and verify hypotheses.
· Manage and report via dashboards and papers the results of daily, weekly, and monthly reporting


Basic Qualifications
· Bachelor’s Degree in Computer Science or a related technical field.
· 6+ years of experience developing data management systems, tools and architectures using SQL, databases, Redshift and/or other distributed computing systems.
· Familiarity with new advances in the data engineering space such as EMR and NoSQL technologies like Dynamo DB.
· Experience designing and operating very large Data Warehouses.
· Demonstrated strong data modelling skills in areas such as data mining and machine learning.
· Proficient in Oracle, Linux, and programming languages such as R, Python, Ruby or Java.
· Skilled in presenting findings, metrics and business information to a broad audience consisting of multiple disciplines and all levels or the organizations.
· Track record for quickly learning new technologies.
· Solid experience in at least one business intelligence reporting tool, e.g. Tableau.
· An ability to work in a fast-paced environment where continuous innovation is occurring and ambiguity is the norm.


Preferred Qualification
· Master’s degree in Information Systems or a related field.
· Capable of investigating, familiarizing and mastering new datasets quickly.
· Knowledge of a programming or scripting language (R, Python, Ruby, or JavaScript).
· Experience with MPP databases such as Greenplum, Vertica, or Redshift
· Experience with Java and Map Reduce frameworks such as Hive/Hadoop.
· 1+ years of experience managing an Analytic or Data Engineering team.
· Strong organizational and multitasking skills with ability to balance competing priorities.



Oaktable World 2016 Sept 19 & 20 is on !!

August 26th, 2016

Having take a new job at Amazon just two weeks ago and moving to Seattle (!)  I didn’t have the time nor was it practical to set up Oaktable World  this year. Luckily Kellyn Pot’vin has taken over the mantel !

Get the info straight from the horses mouth at her blog.

The following content has been graciously supplied from Kellyn’s blog:

Oak Table World is FREE to the PUBLIC!  We don’t require an Oracle Open World badge to attend, so bring a friend and they’ll owe you big time!

otwHere is the current schedule:

The Great Dane

Mogens Norgaard will be opening Oak Table World on

  • Monday, at 8:30am.

Be prepared to be awe-inspired by all he has to share with you, (which may go hand-in-hand with the amount of coffee we can provide to him…)






Denial of Service (DoS) attacks continue

August 24th, 2016

It’s frustrating to have to spend time jumping off into web security and wordpress configurations when there are so many other things that are important to be doing. Today the DoS continued and the Jetpack solution didn’t seem to work. The other two solutions from Digital Ocean didn’t seem reasonable. One was to re-install wordpress with there install version. Nice that they offer a better security protected version but I didn’t feel like re-installing my wordpress. TH other option basically eliminated all access to xmlrpc.php. Looking around I found a plugin that does firewall work and had just added functionality for the xmlrpc.php problem, called ninjafirewall.

Problem is after I installed it I was getting 500 ” Internal server error” errors trying to access this blog.

Turns out the solution is to add a few lines to /etc/apache2/apache2.conf

such as

<Directory /var/www.kyle>

        Options FollowSymLinks

        AllowOverride All


where my WordPress files are hosted in  /var/www.kyle

This didn’t work and I went down many ratholes trying other things. Problem was there was a another line in my  apache2.conf that said

<Directory /var/www>

        Options FollowSymLinks

        AllowOverride FileInfo


I had done some hacking stuff like changed all “AllowOverride None” to “AllowOverride All” but I hadn’t looked for “AllowOverride FileInfo” and second part is that “/var/www” is a link to “/var/www.kyle” thus overriding my “AllowOverride All” . Long story short changing

<Directory /var/www>

        Options FollowSymLinks

        AllowOverride FileInfo



<Directory /var/www>

        Options FollowSymLinks

        AllowOverride All


fixed the problem.

Then I was able to install NinjaFirewall and configure it.

Going to the side bar in WordPress admin view, select “Firewall Policies”

Screen Shot 2016-08-23 at 5.30.01 PM

then select “Block any access to the API” for “WordPress XML-RPC API”Screen Shot 2016-08-23 at 5.30.34 PM

that works. Now the apache log shows 403 errors for access to xmlrpc.php

root@datavirtualizer:/etc/apache2# tail -f /var/log/apache2/other_vhosts_access.log - - [23/Aug/2016:20:31:47 -0400] "POST /xmlrpc.php HTTP/1.1" 403 376 "-" "Googlebot/2.1 (+" - - [23/Aug/2016:20:31:47 -0400] "POST /xmlrpc.php HTTP/1.1" 403 376 "-" "Googlebot/2.1 (+" - - [23/Aug/2016:20:31:48 -0400] "POST /xmlrpc.php HTTP/1.1" 403 376 "-" "Googlebot/2.1 (+" - - [23/Aug/2016:20:31:48 -0400] "POST /xmlrpc.php HTTP/1.1" 403 376 "-" "Googlebot/2.1 (+"

Actually I think an alternative and better method (don’t trust me I don’t fully understand the options) is to leave the xmlrpc stuff off and got to “login protect” . Choose “only under attack”

Screen Shot 2016-08-23 at 5.41.49 PM


for more info.



seems to be the main culprit.

Is there any way to report stuff like this?

IP address

Address type IPv4
ISP Host1plus-cloud-servers
Timezone America/Los_Angeles (UTC-7)
Local time 17:33:55
Country United States   
State / Region Nevada
District / County Clark County
City Las Vegas
Zip / Postal code 89136
Coordinates 36.1146, -115.173




Decided to see what effect the firewall had in the logs.

The logs look like ::1 - - [22/Aug/2016:06:42:05 -0400] "OPTIONS * HTTP/1.0" 200 110 "-" ::1 - - [22/Aug/2016:06:42:05 -0400] "OPTIONS * HTTP/1.0" 200 110 "-" ::1 - - [22/Aug/2016:06:42:06 -0400] "OPTIONS * HTTP/1.0" 200 110 "-" ::1 - - [22/Aug/2016:06:42:06 -0400] "OPTIONS * HTTP/1.0" 200 110 "-" 

so we have the date in field 5 and HTML return code in field 10.

Wrote an awk script to get the date truncated to the hour and count of return code by code type by hour

grep xmlrpc $1 |  \
sed -e 's/:/ /g' | \
sed -e 's/\[/ /g' | \
awk 'BEGIN {
     printf "%14s , ", "date"
     for ( code in codes ) {
       printf "%7s , " , code

     for ( dt in dates ) {
       printf  "\n%14s ", dt
       for ( code in codes ) {
          printf  ", %7i ", cnt[dt,code]+0 ;
   print " "

root@datavirtualizer:/var/log/apache2# ./log.awk other_vhosts_access.log
          date ,     401 ,     403 ,     200 ,     404 ,     500 ,     405 , 
 24/Aug/201600 ,    1757 ,       0 ,     217 ,       0 ,       0 ,       0 
 24/Aug/201601 ,    2833 ,       0 ,      96 ,       0 ,       0 ,       2 
 24/Aug/201602 ,     610 ,       0 ,     502 ,       0 ,       0 ,       1 
 24/Aug/201603 ,     666 ,       0 ,     401 ,       0 ,       0 ,       1 
 24/Aug/201604 ,    1555 ,       0 ,      98 ,       0 ,       0 ,       0 
 24/Aug/201605 ,    2927 ,       0 ,     104 ,       0 ,       0 ,       0 
 24/Aug/201606 ,    3914 ,       0 ,      98 ,       0 ,       0 ,       1 

then plotted in Excel. In Excel just cut and pasted from Unix, chose import wizard and chose comma delineated:


Screen Shot 2016-08-24 at 4.29.55 PM


plotting looks like

Screen Shot 2016-08-24 at 4.20.40 PM


we can see that after applying the firewall at 5pm yesterday , i.e. 17:00 hours, which shows up as  23/Aug/201617 in the x-axis legend, we can see a spike in 403s (forbidden) when I first set up no access to xmlrpc.php and then 401s (unauthorized) after I changed the option to ask for username password after too many access in a few seconds.


Denial of Service (DoS) attack on this site

August 22nd, 2016

This site had been running for a years with no big issues.
I had performance and system saturation issues about 3 years ago and then move the site to is inexpensive and the performance is awesome.

Then last Monday and every day since the site had been going down.
Simplest “solution” for me was just to get on and bounce the machine.

That cleared it up.

After this went on for a few days I contacted DigitalOcean saying I didn’t see how it could be an issue with them, but I’d asked anyway.

Sure enough they had identified the issue, which had nothing to do with them, and gave me the solution.

Read their solution page for more information.

Basically the problem is a DoS attack using xmlrpc.php from WordPress.
To verify this I looked into the logs and sure enough there is rapid access to xmlrpc.php

cd /var/log/apache2

grep xmlrpc *

other_vhosts_access.log.1: – – [14/Aug/2016:10:09:25 -0400] “POST /xmlrpc.php HTTP/1.1″ 500 569 “-” “-”

other_vhosts_access.log.1: – – [14/Aug/2016:10:09:25 -0400] “POST /xmlrpc.php HTTP/1.1″ 500 0 “-” “-”

other_vhosts_access.log.1: – – [14/Aug/2016:10:09:26 -0400] “POST /xmlrpc.php HTTP/1.1″ 500 569 “-” “-“

There are number of solutions, but the easiest for me was to use the Jetpack plugging which comes with a “protect” option. After activating the protect options sure enough the xmlrpc.php access stop. In the following “grep” we see xmlrpc.php rapid access just before the Jetpack option is turned on then they stop. Yay.

root@datavirtualizer:/var/log/apache2# date

Mon Aug 22 12:58:07 EDT 2016

root@datavirtualizer:/var/log/apache2# grep rpc other_vhosts_access.log – – [22/Aug/2016:07:33:28 -0400] “GET /xmlrpc.php HTTP/1.1″ 500 569 “-” “PycURL/7.19.7″ – – [22/Aug/2016:09:01:24 -0400] “POST /xmlrpc.php HTTP/1.1″ 500 569 “” “Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36″ – – [22/Aug/2016:09:01:52 -0400] “POST /xmlrpc.php HTTP/1.1″ 500 569 “” “Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36″ – – [22/Aug/2016:09:17:56 -0400] “GET /xmlrpc.php HTTP/1.1″ 500 569 “-” “PycURL/7.19.5 libcurl/7.38.0 GnuTLS/3.3.8 zlib/1.2.8 libidn/1.29 libssh2/1.4.3 librtmp/2.3″ – – [22/Aug/2016:09:17:59 -0400] “GET /xmlrpc.php HTTP/1.1″ 500 569 “-” “PycURL/7.24.0″ – – [22/Aug/2016:09:18:00 -0400] “GET /xmlrpc.php HTTP/1.1″ 500 569 “-” “PycURL/7.19.5 libcurl/7.38.0 GnuTLS/3.3.8 zlib/1.2.8 libidn/1.29 libssh2/1.4.3 librtmp/2.3″ – – [22/Aug/2016:09:18:03 -0400] “GET /xmlrpc.php HTTP/1.1″ 500 569 “-” “PycURL/7.24.0″ – – [22/Aug/2016:09:40:36 -0400] “GET /xmlrpc.php HTTP/1.1″ 500 569 “-” “PycURL/7.19.7″ – – [22/Aug/2016:12:11:17 -0400] “GET /xmlrpc.php HTTP/1.1″ 500 569 “-” “PycURL/7.19.7″ – – [22/Aug/2016:12:54:25 -0400] “GET /xmlrpc.php?rsd HTTP/1.1″ 200 995 “-” “Mozilla/4.0 (compatible;)” – – [22/Aug/2016:12:54:26 -0400] “GET /xmlrpc.php HTTP/1.1″ 405 281 “-” “Mozilla/4.0 (compatible;)” – – [22/Aug/2016:12:55:56 -0400] “GET /xmlrpc.php?rsd HTTP/1.1″ 200 995 “-” “Mozilla/4.0 (compatible;)” – – [22/Aug/2016:12:55:56 -0400] “GET /xmlrpc.php HTTP/1.1″ 405 281 “-” “Mozilla/4.0 (compatible;)”


Let’s see if the site stays up now.


Speaking at OOW 2016 on Sunday on Active Session History

August 19th, 2016

Session ID: UGF3110

Advanced Active Session History Analytics
Moscone West—3020
09/18/16, 03:30:00 PM – 04:15:00 PM




Moving to Seattle! Our beautiful rental available in San Francisco

June 10th, 2016


More news coming, but for now, we (family and I) are moving to Seattle! We will be giving up our gorgeous rental house in San Francisco thus it will be available to the next lucky family.

Beautiful large house a block from San Francisco’s most prestigious neighborhood St Francis Woods.

see: Photos of house

It is a wonderful elegant big family house for our family and two kids walking distance to Commodore Sloat a block from St Francis Woods on a safe family street. We love it but we have to move because of my job change.

Its an amazing home. Perfect for our family as we walk our kids to school on a walking path. The front yards have no driveways so it’s super safe for kids to play.

I commute both to Palo Alto fast on 280 and I take the muni fast to down town. It’s a secret that the fastest way to down town is on the K, L or M via West Portal because it’s underground. Blows the N-Judah or J-Church away.

It’s a big house just under 3000 square feet. Massive Wolf Stove in kitchen – biggest I’ve seen. Huge beautiful den with working fire place and vaulted ceilings. Beautiful dining room. Hardwood throughout. A beauty.

It can be a 4 bedroom – we used 3 for ourselves and one for a nanny.
It has 2.5 baths meaning one bathroom with shower and separate bath and toilet, a separate toilet on first floor and downstairs a shower and toilet.

Full laundry downstairs. Massive garage –  3 car parking and workshop and lots of storage 

No pets or smoking. As far as I understand it, the neighborhood HOAs don’t allow roommate situations thus in my mind it makes sense for families.

Hardwood floors throughout. 

Fenced yard, ocean view.

Excellent public schools (Top rated in SF with neighborhood preference) Top rated private schools nearby: St.Brendans, St.Stevens, Stratford 

Beautiful walk to West Portal shops, restaurants, parks, library, movie theatre…

The best SF has to offer of calm and families neighborhood with quick access to the hustle and bussle


The rent is $5350

1 Year Lease

Photos of house

Craiglist add:

**We are the current tenants and have loved living in this amazing home and neighborhood.

27309981430_568d32b356_z 27514129171_798a4da473_z 27552759266_378712c56f_z