섹션 16 AWS 스토리지 추가기능

Snow Family :오프라인에서 데이터 마이크레이션을 실행하는 장치

snowball edge

• physical data transport solution : move TB, PBs of data in or out of AWS
• Pay per data transfer job
• snowball edge storage optimized
• snowball edge compute optimized
• 컴퓨팅 능력을 갖추고 있고, 데이터가 snowball로 이동하는 동안 데이터를 사전에 처리할수 있다.

snowcone

컴퓨팅 가능한 작은 장치

snowmobile

트럭

• transfer exabytes of data (1EB = 1000PB =1000,000TB)

Edge Computing

인터넷 연결이 제한되거나 안되는곳에 필요하다.

snowball into Glacier

• snowball cannot import to glacier directly
• s3 first
• snowball → amazon S3 → Amazon glacier

Amazon FSx

• launch 3rd party high-performance file systems on AWS
• Fully managed service

1. FSx for Lustre
2. FSx for NetApp ONTAP
3. FSx for windows file server
4. FSx for OpenZFS

4개의 서비스 차이를 알고 언제 사용하는지 알아야 함

FSx for windows file server

• FSx for windows is a fully managed windows file system share drive
• can be mounted on Linux EC2 instances

FSx for Lustre

lustre = linux + cluster

높은 연산에 사용됨

Seamless integration with S3

• can “read s3” as a file system
• can write the output of the computations back to S3
• can be used from on-promises servers
• 연구를 수행하기 위해 IOPS 를 최대화해 줄 분산 POSIX 준수 파일시스템에 사용 가능

FSx file system Deployment options

• Scratch File system
  • temporary storage
  • data is not replicated
  • high burst
  • usuage : short-term processing ,optimize costs
• persistent File system
  • long-term storage
  • data is replicated within same AZ
  • replace failed files within minutes
  • long-term processing

FSx for NetApp ONTAP

move workloads running on ONTAP or NAS to AWS

Storage shrinks or grows automatically

snapshots , replication ,low-cost , compression and data de-duplication

point-in -time instantaneous cloning

FSx for OpenZFS

• file system compatible with NFS
• move workloads running on ZFS to AWS

AWS Storage Gateway

• bridge between on-premises data and cloud data
• use cases:
  • disaster recovery
  • backup & restore
  • tired storage

S3 file Gateway

• configured S3 buckets are accessible using the NFS and SMB protocol
• Most recently used data is cached in the file gateway
• supports S3 standard , s3 standard IA , S3 one Zone A, S3 intelligent Tiering
• transition to S3 glacier using a lifecycle policy

FSx File Gateway

local cache for frequently accessed data

S3에 대규모의 데이터셋이저장 되있고 NFS 혹은 SMB 프로토콜을 사용해 온프레미스 서버를 통해 이 데이터셋에 액세스 하려고 한다.

온프레미스 microsoft AD를 통해 이 파일에 대한 액세스를 인증하고 할때 사용 가능

Volume Gateway

block storage using iSCSI protocol backed by s3

backed by EBS snaphots which can help restore on-premises volumes

• cached volumes : low latency access to most recent data
• stored volumes : entire dataset is on premise ,scheduled backups

Tape Gateway

백업 시스템이 있는 회사가 백업에 테이프 대신에 클라우드를 활용하는것

테이프 백업에 가상 인피니트 스토리지를 노출하려고 한다.

동일한 소프트웨어를 유지하고 ,iSCSI 와 호환 가능한 인터페이스

AWS Transfer Family

• fully -managed service for file transfers into and out of Amazon S3 or Amazon EFS using the FTP protocol
• AWS Transfer for FTP ( File transfer protocol)
• AWS Transfer for FTPS(file transfer protocol over SSL )
• AWS Transfer for SFTP (secure file transfer protocol )

FTP 는 암호화 안됨 나머지 두개는 암호화

AWS DataSync

• 대용량 데이터를 다른곳으로 옮길수 있음
• on-premises → other cloud to AWS
• AWS to AWS
• File permissions and metadata are preserved
• DataSync 을 사용할때 네트워크 용량이 딸리는 경우 aws snowcone 사용

storage comparision

• s3: object stroage
• s3 glacier : object archival
• EBS volumes : network storage for one EC2 instance ar a time
• instance stroage : physical storage for your EC2 instance (high IOPS)
• EFS : Network file system for linux instances , POSIX filesystem
• FSx for Windows: network file system for windows servers
• FSx for Lustre : high performance computing linux file system
• FSx for NetApp ONTAP : High os compatibility
• FSx for OpenZFS : Managed ZFS file system
• storage gateway : S3 & FSx file gateway, volume gateway ,tape gateway
• transfer family : FTP , FTPS,SFTP
• DataSync : schedule data sync from on-premises to AWS, or AWS to AWS
• Snowcone / snowball /snowmobile : to move large amount of data to the cloud
• database : for specific worloads ,usually with indexing and querying